2024-04-25T11:49:36.215 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T11:49:36.215 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/norstats}} --verbose --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340 -- /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340/orig.config.yaml 2024-04-25T11:49:36.275 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:49:36.383 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/norstats}} email: null first_in_suite: false job_id: '7673340' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client dirsize rbytes: false client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync rbytes: false syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: df40303764f976ac2e973c1ef2b40d1ebbda473b owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2866 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: df40303764f976ac2e973c1ef2b40d1ebbda473b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLIwP2JCCUCyS+uAqsSjt6vJswyEJitzVChNFDE5fYTzW51GPUekb5nTMdmGVdAzHn0BKOp7a5mf2EAtctxln9E= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKKZLkrHzCWxp/EkCzdYWBohZ6e2yWtEhdEyM2rlc1W/W/BoLZ/cnw5kFdvj58/bez8xOVi3OsX8jUAydC+ENkE= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLYgstvRDX4cVmMg8yvJKTYMMudY1X4BY3Bx1vUXf/U5gvj/TeREmehA49fHgA1Nh1OISpZqjMSe65z+dBypyYA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/norstats teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_11:13:36 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T11:49:36.384 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa; will attempt to use it 2024-04-25T11:49:36.384 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa/tasks 2024-04-25T11:49:36.385 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T11:49:36.386 INFO:teuthology.task.internal:Checking packages... 2024-04-25T11:49:36.432 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T11:49:36.432 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T11:49:36.432 INFO:teuthology.packaging:ref: None 2024-04-25T11:49:36.432 INFO:teuthology.packaging:tag: None 2024-04-25T11:49:36.432 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T11:49:36.432 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:49:36.432 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T11:49:36.591 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T11:49:36.593 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T11:49:36.604 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T11:49:36.604 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T11:49:36.654 INFO:teuthology.task.internal:Saving configuration 2024-04-25T11:49:36.676 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T11:49:36.689 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T11:49:36.714 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:44:18.494761', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLIwP2JCCUCyS+uAqsSjt6vJswyEJitzVChNFDE5fYTzW51GPUekb5nTMdmGVdAzHn0BKOp7a5mf2EAtctxln9E='} 2024-04-25T11:49:36.737 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:44:18.497801', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKKZLkrHzCWxp/EkCzdYWBohZ6e2yWtEhdEyM2rlc1W/W/BoLZ/cnw5kFdvj58/bez8xOVi3OsX8jUAydC+ENkE='} 2024-04-25T11:49:36.761 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:44:18.496415', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLYgstvRDX4cVmMg8yvJKTYMMudY1X4BY3Bx1vUXf/U5gvj/TeREmehA49fHgA1Nh1OISpZqjMSe65z+dBypyYA='} 2024-04-25T11:49:36.761 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T11:49:36.772 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T11:49:36.773 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T11:49:36.773 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T11:49:36.773 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T11:49:36.901 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9e6d6e50d0>, signals=[15]) 2024-04-25T11:49:36.901 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T11:49:36.910 INFO:teuthology.task.internal:Opening connections... 2024-04-25T11:49:36.910 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T11:49:36.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:49:37.031 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T11:49:37.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:49:37.106 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T11:49:37.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:49:37.181 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T11:49:37.190 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-04-25T11:49:37.209 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-04-25T11:49:37.209 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-04-25T11:49:37.267 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-04-25T11:49:37.267 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-04-25T11:49:37.267 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-04-25T11:49:37.267 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-04-25T11:49:37.267 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-04-25T11:49:37.267 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-04-25T11:49:37.267 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T11:49:37.267 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-04-25T11:49:37.268 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-04-25T11:49:37.268 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T11:49:37.268 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-04-25T11:49:37.268 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T11:49:37.268 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T11:49:37.268 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T11:49:37.268 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-04-25T11:49:37.289 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-04-25T11:49:37.307 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-04-25T11:49:37.307 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-04-25T11:49:37.364 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-04-25T11:49:37.364 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-04-25T11:49:37.364 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-04-25T11:49:37.364 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-04-25T11:49:37.364 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-04-25T11:49:37.364 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-04-25T11:49:37.364 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T11:49:37.365 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-04-25T11:49:37.365 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-04-25T11:49:37.365 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T11:49:37.365 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-04-25T11:49:37.365 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T11:49:37.365 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T11:49:37.365 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T11:49:37.365 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-04-25T11:49:37.389 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-04-25T11:49:37.407 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-04-25T11:49:37.407 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-04-25T11:49:37.465 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-04-25T11:49:37.465 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-04-25T11:49:37.465 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-04-25T11:49:37.465 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-04-25T11:49:37.465 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-04-25T11:49:37.465 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-04-25T11:49:37.465 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T11:49:37.465 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-04-25T11:49:37.466 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-04-25T11:49:37.466 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T11:49:37.466 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-04-25T11:49:37.466 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T11:49:37.466 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T11:49:37.466 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T11:49:37.466 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-04-25T11:49:37.481 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T11:49:37.492 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T11:49:37.526 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T11:49:37.526 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:49:37.529 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:49:37.531 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:49:37.547 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T11:49:37.557 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T11:49:37.557 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:49:37.586 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:49:37.588 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:49:37.647 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T11:49:37.724 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T11:49:37.741 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T11:49:37.741 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T11:49:37.741 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T11:49:37.741 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:49:37.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:49:37.759 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-04-25T11:49:37.817 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-437.el9.x86_64 2024-04-25T11:49:37.817 INFO:teuthology.task.kernel:Running kernel on smithi019: 5.14.0-437.el9.x86_64 2024-04-25T11:49:37.817 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2024-04-25T11:49:40.354 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.0 MB 00:00 2024-04-25T11:49:42.804 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-25T11:49:47.414 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 6.2 MB 00:00 2024-04-25T11:49:49.124 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-25T11:49:50.202 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-25T11:49:55.458 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T11:49:57.439 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T11:49:57.439 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T11:49:57.541 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T11:49:57.542 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-04-25T11:49:57.542 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-25T11:49:57.653 DEBUG:teuthology.orchestra.run.smithi019:> echo no | sudo yum reinstall kernel || true 2024-04-25T11:49:58.176 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 11:49:55 AM UTC. 2024-04-25T11:49:58.312 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T11:49:58.320 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T11:49:58.320 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-04-25T11:49:58.321 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T11:49:58.321 INFO:teuthology.orchestra.run.smithi019.stdout:Reinstalling: 2024-04-25T11:49:58.321 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T11:49:58.321 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:49:58.321 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-04-25T11:49:58.321 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T11:49:58.321 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:49:58.321 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 255 k 2024-04-25T11:49:58.321 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 0 2024-04-25T11:49:58.321 INFO:teuthology.orchestra.run.smithi019.stderr:Operation aborted. 2024-04-25T11:49:58.379 INFO:teuthology.orchestra.run.smithi019.stdout:Is this ok [y/N]: 2024-04-25T11:49:58.379 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum reinstall -y kernel || true 2024-04-25T11:49:58.899 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 11:49:55 AM UTC. 2024-04-25T11:49:59.020 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T11:49:59.020 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T11:49:59.020 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-04-25T11:49:59.021 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T11:49:59.021 INFO:teuthology.orchestra.run.smithi019.stdout:Reinstalling: 2024-04-25T11:49:59.021 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T11:49:59.021 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:49:59.021 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-04-25T11:49:59.021 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-04-25T11:49:59.021 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:49:59.021 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 255 k 2024-04-25T11:49:59.021 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 0 2024-04-25T11:49:59.021 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-04-25T11:49:59.368 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.1 MB/s | 255 kB 00:00 2024-04-25T11:49:59.368 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:49:59.368 INFO:teuthology.orchestra.run.smithi019.stdout:Total 735 kB/s | 255 kB 00:00 2024-04-25T11:49:59.388 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-04-25T11:49:59.568 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-04-25T11:49:59.568 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-04-25T11:49:59.573 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-04-25T11:49:59.573 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-04-25T11:49:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-04-25T11:49:59.785 INFO:teuthology.orchestra.run.smithi019.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T11:49:59.785 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:50:00.410 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:50:00.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T11:50:01.264 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:50:01.264 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:50:01.264 INFO:teuthology.orchestra.run.smithi019.stdout:Reinstalled: 2024-04-25T11:50:01.265 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T11:50:01.265 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:50:01.265 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-25T11:50:01.445 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T11:50:01.494 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T11:50:01.495 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T11:50:01.495 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T11:50:01.495 DEBUG:teuthology.parallel:result is None 2024-04-25T11:50:01.495 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T11:50:01.503 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T11:50:01.503 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:50:01.506 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:50:01.508 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:50:01.526 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T11:50:01.542 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T11:50:01.607 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T11:50:01.608 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:50:01.610 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:50:01.612 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:50:01.699 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T11:50:01.707 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T11:50:01.707 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:50:01.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:50:01.725 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:50:01.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:50:01.741 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:50:01.744 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:50:01.768 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:50:01.777 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:50:01.806 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:50:01.809 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:50:01.818 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:50:01.848 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:50:01.850 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:50:01.851 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T11:50:01.858 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T11:50:01.859 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:50:01.861 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:50:01.892 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:50:01.919 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T11:50:01.928 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T11:50:01.929 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:50:01.931 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:50:01.960 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:50:01.975 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:50:02.027 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:50:02.074 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:50:02.152 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:50:02.196 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:50:02.196 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:50:02.260 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:50:02.306 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:50:02.357 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:50:02.438 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:50:02.484 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T11:50:02.484 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:50:02.550 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:50:02.595 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:50:02.658 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:50:02.702 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:50:02.743 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:50:02.743 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:50:02.809 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-04-25T11:50:02.812 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-04-25T11:50:02.814 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-04-25T11:50:02.850 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:50:02.852 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:50:02.886 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:50:03.325 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T11:50:03.333 INFO:teuthology.task.internal:Starting timer... 2024-04-25T11:50:03.333 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T11:50:03.376 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T11:50:03.399 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T11:50:03.399 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-04-25T11:50:03.462 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-04-25T11:50:03.464 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-04-25T11:50:03.527 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-04-25T11:50:03.529 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2024-04-25T11:50:03.591 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2024-04-25T11:50:03.593 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T11:50:03.593 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-04-25T11:50:03.618 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-04-25T11:50:03.618 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-04-25T11:50:03.641 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-04-25T11:50:03.642 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2024-04-25T11:50:03.666 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2024-04-25T11:50:03.666 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive'} 2024-04-25T11:50:03.667 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:50:03.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:50:03.693 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:50:03.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:50:03.721 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:50:03.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:50:03.745 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T11:50:03.745 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:50:03.770 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:50:03.796 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:50:03.821 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T11:50:03.830 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T11:50:03.835 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T11:50:03.966 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T11:50:03.981 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T11:50:03.983 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi150.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T11:53:53.459 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2024-04-25T11:53:53.461 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T11:53:53.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:53:53.544 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-04-25T11:53:53.630 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-04-25T11:53:53.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-25T11:53:53.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:53:53.705 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-04-25T11:53:53.789 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-25T11:53:53.789 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T11:53:53.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:53:53.872 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-04-25T11:53:53.950 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-25T11:53:53.951 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T11:53:53.964 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T11:53:53.964 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:53:53.965 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:53:53.967 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:53:53.967 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:53:53.969 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:53:53.969 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:53:53.995 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:53:53.997 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:53:54.009 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:53:54.013 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:53:54.019 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:53:54.031 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-04-25T11:53:54.033 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:53:54.038 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-04-25T11:53:54.043 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-04-25T11:53:54.050 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-04-25T11:53:54.054 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:53:54.057 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2024-04-25T11:53:54.065 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:53:54.067 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2024-04-25T11:53:54.067 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:53:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:53:54.080 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:53:54.093 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:53:54.127 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:53:54.131 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:53:54.131 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-04-25T11:53:54.131 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:53:54.131 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:53:54.131 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:53:54.131 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:53:54.134 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:53:54.139 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:53:54.139 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-04-25T11:53:54.139 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:53:54.139 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:53:54.139 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:53:54.139 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:53:54.153 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:53:54.156 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:53:54.156 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-25T11:53:54.156 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:53:54.156 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:53:54.156 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:53:54.156 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:53:54.157 INFO:teuthology.run_tasks:Running task install... 2024-04-25T11:53:54.166 DEBUG:teuthology.task.install:project ceph 2024-04-25T11:53:54.166 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T11:53:54.166 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T11:53:54.167 INFO:teuthology.task.install:Using flavor: default 2024-04-25T11:53:54.173 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T11:53:54.173 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T11:53:54.173 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T11:53:54.174 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:53:54.176 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T11:53:54.177 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:53:54.178 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T11:53:54.178 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:53:54.327 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T11:53:54.327 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T11:53:54.399 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T11:53:54.399 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T11:53:54.412 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T11:53:54.413 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T11:53:54.455 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:53:54.455 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T11:53:54.455 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:53:54.481 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:53:54.481 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:53:54.525 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:53:54.525 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:53:54.525 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:53:54.547 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:53:54.552 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:53:54.552 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:53:54.574 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:53:54.575 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:53:54.575 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:53:54.601 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:53:54.602 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:53:54.616 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:53:54.620 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:53:54.668 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:53:54.683 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-04-25T11:53:54.684 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-04-25T11:53:54.691 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:53:54.744 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:53:54.754 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2024-04-25T11:53:54.756 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2024-04-25T11:53:54.809 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-04-25T11:53:54.810 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-04-25T11:53:54.906 INFO:teuthology.orchestra.run.smithi150.stdout:48 files removed 2024-04-25T11:53:54.934 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:53:54.980 INFO:teuthology.orchestra.run.smithi169.stdout:48 files removed 2024-04-25T11:53:55.007 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:53:55.044 INFO:teuthology.orchestra.run.smithi019.stdout:48 files removed 2024-04-25T11:53:55.071 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:53:55.658 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 181 kB/s | 83 kB 00:00 2024-04-25T11:53:55.677 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 204 kB/s | 83 kB 00:00 2024-04-25T11:53:55.681 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 247 kB/s | 83 kB 00:00 2024-04-25T11:53:56.045 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-04-25T11:53:56.100 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-25T11:53:56.107 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-25T11:53:56.440 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 6.4 kB/s | 1.9 kB 00:00 2024-04-25T11:53:56.495 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-04-25T11:53:56.534 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-04-25T11:53:57.533 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.0 MB 00:00 2024-04-25T11:53:59.312 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.0 MB 00:02 2024-04-25T11:53:59.510 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T11:53:59.815 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 8.0 MB 00:03 2024-04-25T11:54:01.263 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T11:54:01.866 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-25T11:54:03.894 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 6.2 MB 00:00 2024-04-25T11:54:05.355 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T11:54:05.578 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-25T11:54:06.509 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 6.2 MB 00:00 2024-04-25T11:54:06.696 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-04-25T11:54:07.006 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-25T11:54:07.917 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 27 MB/s | 21 MB 00:00 2024-04-25T11:54:07.968 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 16 kB 00:00 2024-04-25T11:54:11.439 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 21 MB 00:03 2024-04-25T11:54:11.855 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 6.5 kB/s | 1.7 kB 00:00 2024-04-25T11:54:13.008 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T11:54:13.982 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:54:13.982 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:54:13.993 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:54:13.994 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:54:13.995 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:54:13.996 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:54:13.997 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:54:13.999 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:54:14.000 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:54:14.062 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-25T11:54:14.084 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-25T11:54:14.084 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-25T11:54:14.084 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-25T11:54:14.084 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-25T11:54:14.085 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:54:14.085 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T11:54:14.085 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:54:14.085 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T11:54:14.085 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T11:54:14.085 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T11:54:14.085 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T11:54:14.085 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T11:54:14.085 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T11:54:14.085 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T11:54:14.085 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T11:54:14.085 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T11:54:14.086 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T11:54:14.086 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T11:54:14.086 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T11:54:14.086 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T11:54:14.086 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T11:54:14.086 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:54:14.086 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:54:14.086 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:54:14.086 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T11:54:14.086 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T11:54:14.086 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T11:54:14.086 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:54:14.087 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:54:14.087 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:54:14.087 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:54:14.087 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:54:14.087 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:54:14.087 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T11:54:14.087 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T11:54:14.087 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T11:54:14.087 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T11:54:14.087 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T11:54:14.087 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:54:14.087 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T11:54:14.087 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:54:14.088 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-04-25T11:54:14.088 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T11:54:14.088 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:54:14.088 INFO:teuthology.orchestra.run.smithi150.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:54:14.088 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:54:14.088 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-25T11:54:14.088 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:54:14.088 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:54:14.088 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:54:14.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T11:54:14.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T11:54:14.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T11:54:14.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T11:54:14.089 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T11:54:14.089 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T11:54:14.089 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T11:54:14.089 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T11:54:14.089 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:54:14.089 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:54:14.089 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:54:14.089 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:54:14.089 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:54:14.089 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:54:14.089 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:54:14.089 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:54:14.089 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T11:54:14.090 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:54:14.090 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:54:14.090 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:54:14.090 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:54:14.090 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:54:14.090 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T11:54:14.090 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:54:14.090 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:54:14.090 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:54:14.090 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:54:14.090 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:54:14.090 INFO:teuthology.orchestra.run.smithi150.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:54:14.090 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:54:14.091 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:54:14.091 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:54:14.091 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:54:14.091 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:54:14.091 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:54:14.091 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:54:14.091 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:54:14.091 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:54:14.091 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:54:14.091 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:54:14.091 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:54:14.091 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:54:14.091 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T11:54:14.091 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T11:54:14.092 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:54:14.092 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:54:14.092 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:54:14.092 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:54:14.092 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:54:14.092 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:54:14.092 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:54:14.092 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:54:14.092 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:54:14.092 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:54:14.092 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:54:14.092 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:54:14.092 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:54:14.093 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:54:14.093 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:54:14.093 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:54:14.093 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:54:14.093 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:54:14.093 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:54:14.093 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:54:14.093 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:54:14.093 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:54:14.093 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:54:14.093 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:54:14.093 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:54:14.093 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:54:14.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:54:14.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:54:14.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:54:14.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:54:14.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:54:14.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:54:14.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:54:14.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:54:14.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:54:14.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:54:14.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:54:14.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:54:14.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:54:14.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:54:14.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:54:14.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:54:14.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:54:14.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:54:14.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:54:14.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:54:14.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:54:14.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:54:14.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:54:14.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:54:14.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:54:14.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:54:14.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:54:14.096 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:54:14.096 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:54:14.096 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:54:14.096 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:54:14.096 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:54:14.096 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:54:14.096 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:54:14.096 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:54:14.096 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:54:14.096 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:54:14.096 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-04-25T11:54:14.096 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:54:14.096 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:54:14.096 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-25T11:54:14.097 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-25T11:54:14.097 INFO:teuthology.orchestra.run.smithi150.stdout:Install 148 Packages 2024-04-25T11:54:14.097 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 4 Packages 2024-04-25T11:54:14.097 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:54:14.097 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 222 M 2024-04-25T11:54:14.097 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-25T11:54:14.814 INFO:teuthology.orchestra.run.smithi150.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-25T11:54:15.102 INFO:teuthology.orchestra.run.smithi150.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 3.0 MB/s | 890 kB 00:00 2024-04-25T11:54:15.143 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:54:15.143 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:54:15.154 INFO:teuthology.orchestra.run.smithi169.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:54:15.155 INFO:teuthology.orchestra.run.smithi169.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:54:15.156 INFO:teuthology.orchestra.run.smithi169.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:54:15.156 INFO:teuthology.orchestra.run.smithi169.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:54:15.158 INFO:teuthology.orchestra.run.smithi169.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:54:15.160 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:54:15.161 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:54:15.165 INFO:teuthology.orchestra.run.smithi150.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.2 MB/s | 145 kB 00:00 2024-04-25T11:54:15.221 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-04-25T11:54:15.242 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T11:54:15.242 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-04-25T11:54:15.242 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T11:54:15.242 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-04-25T11:54:15.242 INFO:teuthology.orchestra.run.smithi169.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:54:15.243 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T11:54:15.243 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:54:15.243 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T11:54:15.243 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T11:54:15.243 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T11:54:15.243 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T11:54:15.243 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T11:54:15.243 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T11:54:15.243 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T11:54:15.243 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T11:54:15.243 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T11:54:15.243 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T11:54:15.243 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T11:54:15.244 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T11:54:15.244 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T11:54:15.244 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T11:54:15.244 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:54:15.244 INFO:teuthology.orchestra.run.smithi169.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:54:15.244 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:54:15.244 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T11:54:15.244 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T11:54:15.244 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T11:54:15.244 INFO:teuthology.orchestra.run.smithi169.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:54:15.244 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:54:15.244 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:54:15.244 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:54:15.245 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:54:15.245 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:54:15.245 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T11:54:15.245 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T11:54:15.245 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T11:54:15.245 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T11:54:15.245 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T11:54:15.245 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:54:15.245 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T11:54:15.245 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:54:15.245 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-04-25T11:54:15.245 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T11:54:15.245 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:54:15.246 INFO:teuthology.orchestra.run.smithi169.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:54:15.246 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:54:15.246 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-04-25T11:54:15.246 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:54:15.246 INFO:teuthology.orchestra.run.smithi169.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:54:15.246 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:54:15.246 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T11:54:15.246 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T11:54:15.246 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T11:54:15.246 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T11:54:15.246 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T11:54:15.246 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T11:54:15.246 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T11:54:15.247 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T11:54:15.247 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:54:15.247 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:54:15.247 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:54:15.247 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:54:15.247 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:54:15.247 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:54:15.247 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:54:15.247 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:54:15.247 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T11:54:15.247 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:54:15.247 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:54:15.247 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:54:15.248 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:54:15.248 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:54:15.248 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T11:54:15.248 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:54:15.248 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:54:15.248 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:54:15.248 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:54:15.248 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:54:15.248 INFO:teuthology.orchestra.run.smithi169.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:54:15.248 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:54:15.248 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:54:15.248 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:54:15.248 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:54:15.249 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:54:15.249 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:54:15.249 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:54:15.249 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:54:15.249 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:54:15.249 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:54:15.249 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:54:15.249 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:54:15.249 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:54:15.249 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T11:54:15.249 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T11:54:15.249 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:54:15.249 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:54:15.250 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:54:15.250 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:54:15.250 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:54:15.250 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:54:15.250 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:54:15.250 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:54:15.250 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:54:15.250 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:54:15.250 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:54:15.250 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:54:15.250 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:54:15.250 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:54:15.250 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:54:15.251 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:54:15.251 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:54:15.251 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:54:15.251 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:54:15.251 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:54:15.251 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:54:15.251 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:54:15.251 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:54:15.251 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:54:15.251 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:54:15.251 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:54:15.251 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:54:15.251 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:54:15.252 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:54:15.252 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:54:15.252 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:54:15.252 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:54:15.252 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:54:15.252 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:54:15.252 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:54:15.252 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:54:15.252 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:54:15.252 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:54:15.252 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:54:15.252 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:54:15.252 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:54:15.253 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:54:15.253 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:54:15.253 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:54:15.253 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:54:15.253 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:54:15.253 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:54:15.253 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:54:15.253 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:54:15.253 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:54:15.253 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:54:15.253 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:54:15.253 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:54:15.253 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:54:15.254 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:54:15.254 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:54:15.254 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:54:15.254 INFO:teuthology.orchestra.run.smithi169.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:54:15.254 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:54:15.254 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:54:15.254 INFO:teuthology.orchestra.run.smithi169.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:54:15.254 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:54:15.254 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:54:15.254 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2024-04-25T11:54:15.254 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:54:15.254 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:54:15.254 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-04-25T11:54:15.255 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-04-25T11:54:15.255 INFO:teuthology.orchestra.run.smithi169.stdout:Install 148 Packages 2024-04-25T11:54:15.255 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 4 Packages 2024-04-25T11:54:15.255 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:54:15.255 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 222 M 2024-04-25T11:54:15.255 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-04-25T11:54:15.490 INFO:teuthology.orchestra.run.smithi150.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 7.3 MB/s | 2.4 MB 00:00 2024-04-25T11:54:15.649 INFO:teuthology.orchestra.run.smithi150.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 9.4 MB/s | 1.5 MB 00:00 2024-04-25T11:54:16.024 INFO:teuthology.orchestra.run.smithi150.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 13 MB/s | 4.7 MB 00:00 2024-04-25T11:54:16.108 INFO:teuthology.orchestra.run.smithi150.stdout:(7/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 3.7 MB/s | 5.4 MB 00:01 2024-04-25T11:54:16.209 INFO:teuthology.orchestra.run.smithi169.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 33 kB/s | 6.6 kB 00:00 2024-04-25T11:54:16.653 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T11:54:17.035 INFO:teuthology.orchestra.run.smithi169.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.1 MB/s | 890 kB 00:00 2024-04-25T11:54:17.193 INFO:teuthology.orchestra.run.smithi169.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 915 kB/s | 145 kB 00:00 2024-04-25T11:54:17.375 INFO:teuthology.orchestra.run.smithi150.stdout:(8/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 12 MB/s | 17 MB 00:01 2024-04-25T11:54:17.441 INFO:teuthology.orchestra.run.smithi150.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 380 kB/s | 25 kB 00:00 2024-04-25T11:54:18.044 INFO:teuthology.orchestra.run.smithi169.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 2.8 MB/s | 2.4 MB 00:00 2024-04-25T11:54:18.342 INFO:teuthology.orchestra.run.smithi150.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 5.7 MB/s | 21 MB 00:03 2024-04-25T11:54:18.450 INFO:teuthology.orchestra.run.smithi150.stdout:(11/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 4.5 MB/s | 11 MB 00:02 2024-04-25T11:54:18.460 INFO:teuthology.orchestra.run.smithi169.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.6 MB/s | 1.5 MB 00:00 2024-04-25T11:54:18.475 INFO:teuthology.orchestra.run.smithi150.stdout:(12/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.6 MB/s | 225 kB 00:00 2024-04-25T11:54:18.518 INFO:teuthology.orchestra.run.smithi150.stdout:(13/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 404 kB/s | 27 kB 00:00 2024-04-25T11:54:18.543 INFO:teuthology.orchestra.run.smithi150.stdout:(14/152): libcephfs-devel-19.0.0-3156.gd69f13d8 472 kB/s | 32 kB 00:00 2024-04-25T11:54:18.635 INFO:teuthology.orchestra.run.smithi150.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.7 MB/s | 163 kB 00:00 2024-04-25T11:54:18.727 INFO:teuthology.orchestra.run.smithi150.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.4 MB/s | 126 kB 00:00 2024-04-25T11:54:18.777 INFO:teuthology.orchestra.run.smithi150.stdout:(17/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 2.8 MB/s | 747 kB 00:00 2024-04-25T11:54:18.827 INFO:teuthology.orchestra.run.smithi150.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 4.9 MB/s | 500 kB 00:00 2024-04-25T11:54:18.869 INFO:teuthology.orchestra.run.smithi169.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.9 MB/s | 5.4 MB 00:02 2024-04-25T11:54:18.903 INFO:teuthology.orchestra.run.smithi150.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 600 kB/s | 45 kB 00:00 2024-04-25T11:54:18.932 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:54:18.933 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:54:18.944 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:54:18.945 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:54:18.946 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:54:18.947 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:54:18.948 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:54:18.951 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:54:18.952 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:54:19.011 INFO:teuthology.orchestra.run.smithi150.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.2 MB/s | 135 kB 00:00 2024-04-25T11:54:19.018 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-04-25T11:54:19.027 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T11:54:19.027 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-04-25T11:54:19.027 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T11:54:19.027 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-04-25T11:54:19.027 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:54:19.027 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T11:54:19.027 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:54:19.027 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T11:54:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T11:54:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T11:54:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T11:54:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T11:54:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T11:54:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T11:54:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T11:54:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T11:54:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T11:54:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T11:54:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T11:54:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T11:54:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T11:54:19.028 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:54:19.029 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:54:19.029 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:54:19.029 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T11:54:19.029 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T11:54:19.029 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T11:54:19.029 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:54:19.029 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:54:19.029 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:54:19.029 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:54:19.029 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:54:19.029 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:54:19.029 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T11:54:19.029 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T11:54:19.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T11:54:19.030 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T11:54:19.030 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T11:54:19.030 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:54:19.030 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T11:54:19.030 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:54:19.030 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-04-25T11:54:19.030 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T11:54:19.030 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:54:19.030 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:54:19.030 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:54:19.030 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-04-25T11:54:19.030 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:54:19.031 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:54:19.031 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:54:19.031 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T11:54:19.031 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T11:54:19.031 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T11:54:19.031 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T11:54:19.031 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T11:54:19.031 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T11:54:19.031 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T11:54:19.031 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T11:54:19.031 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:54:19.031 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:54:19.031 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:54:19.032 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:54:19.032 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:54:19.032 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:54:19.032 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:54:19.032 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:54:19.032 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T11:54:19.032 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:54:19.032 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:54:19.032 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:54:19.032 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:54:19.032 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:54:19.032 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T11:54:19.032 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:54:19.032 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:54:19.033 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:54:19.033 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:54:19.033 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:54:19.033 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:54:19.033 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:54:19.033 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:54:19.033 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:54:19.033 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:54:19.033 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:54:19.033 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:54:19.033 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:54:19.033 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:54:19.034 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:54:19.034 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:54:19.034 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:54:19.034 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:54:19.034 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:54:19.034 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T11:54:19.034 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T11:54:19.034 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:54:19.034 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:54:19.034 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:54:19.034 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:54:19.034 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:54:19.034 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:54:19.035 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:54:19.035 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:54:19.035 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:54:19.035 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:54:19.035 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:54:19.035 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:54:19.035 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:54:19.035 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:54:19.035 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:54:19.035 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:54:19.035 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:54:19.035 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:54:19.035 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:54:19.036 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:54:19.036 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:54:19.036 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:54:19.036 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:54:19.036 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:54:19.036 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:54:19.036 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:54:19.036 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:54:19.036 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:54:19.036 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:54:19.036 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:54:19.037 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:54:19.037 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:54:19.037 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:54:19.037 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:54:19.037 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:54:19.037 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:54:19.037 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:54:19.037 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:54:19.037 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:54:19.037 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:54:19.037 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:54:19.037 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:54:19.038 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:54:19.038 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:54:19.038 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:54:19.038 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:54:19.038 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:54:19.038 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:54:19.038 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:54:19.038 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:54:19.038 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:54:19.038 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:54:19.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:54:19.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:54:19.039 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:54:19.039 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:54:19.039 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:54:19.039 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:54:19.039 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:54:19.039 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:54:19.039 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:54:19.039 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:54:19.039 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:54:19.040 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-04-25T11:54:19.040 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:54:19.040 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:54:19.040 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-04-25T11:54:19.040 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-04-25T11:54:19.040 INFO:teuthology.orchestra.run.smithi019.stdout:Install 148 Packages 2024-04-25T11:54:19.040 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 4 Packages 2024-04-25T11:54:19.040 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:54:19.041 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 222 M 2024-04-25T11:54:19.041 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-04-25T11:54:19.120 INFO:teuthology.orchestra.run.smithi150.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.5 MB/s | 161 kB 00:00 2024-04-25T11:54:19.229 INFO:teuthology.orchestra.run.smithi150.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.9 MB/s | 320 kB 00:00 2024-04-25T11:54:19.345 INFO:teuthology.orchestra.run.smithi150.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.5 MB/s | 298 kB 00:00 2024-04-25T11:54:19.446 INFO:teuthology.orchestra.run.smithi150.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 996 kB/s | 99 kB 00:00 2024-04-25T11:54:19.546 INFO:teuthology.orchestra.run.smithi150.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 854 kB/s | 85 kB 00:00 2024-04-25T11:54:20.021 INFO:teuthology.orchestra.run.smithi150.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 6.5 MB/s | 3.1 MB 00:00 2024-04-25T11:54:20.139 INFO:teuthology.orchestra.run.smithi150.stdout:(27/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 3.9 MB/s | 5.4 MB 00:01 2024-04-25T11:54:20.163 INFO:teuthology.orchestra.run.smithi150.stdout:(28/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.2 MB/s | 170 kB 00:00 2024-04-25T11:54:20.206 INFO:teuthology.orchestra.run.smithi150.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 512 kB/s | 34 kB 00:00 2024-04-25T11:54:20.230 INFO:teuthology.orchestra.run.smithi019.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-25T11:54:20.240 INFO:teuthology.orchestra.run.smithi150.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.8 MB/s | 142 kB 00:00 2024-04-25T11:54:20.261 INFO:teuthology.orchestra.run.smithi169.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.6 MB/s | 4.7 MB 00:01 2024-04-25T11:54:21.189 INFO:teuthology.orchestra.run.smithi150.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 7.8 MB/s | 7.4 MB 00:00 2024-04-25T11:54:21.254 INFO:teuthology.orchestra.run.smithi019.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 868 kB/s | 890 kB 00:01 2024-04-25T11:54:21.281 INFO:teuthology.orchestra.run.smithi150.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.7 MB/s | 249 kB 00:00 2024-04-25T11:54:21.373 INFO:teuthology.orchestra.run.smithi150.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 538 kB/s | 49 kB 00:00 2024-04-25T11:54:21.413 INFO:teuthology.orchestra.run.smithi019.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 916 kB/s | 145 kB 00:00 2024-04-25T11:54:21.473 INFO:teuthology.orchestra.run.smithi150.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 165 kB/s | 16 kB 00:00 2024-04-25T11:54:21.574 INFO:teuthology.orchestra.run.smithi150.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.7 MB/s | 273 kB 00:00 2024-04-25T11:54:21.632 INFO:teuthology.orchestra.run.smithi150.stdout:(36/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 2.5 MB/s | 3.5 MB 00:01 2024-04-25T11:54:21.691 INFO:teuthology.orchestra.run.smithi150.stdout:(37/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 6.4 MB/s | 762 kB 00:00 2024-04-25T11:54:21.734 INFO:teuthology.orchestra.run.smithi150.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 182 kB/s | 18 kB 00:00 2024-04-25T11:54:21.800 INFO:teuthology.orchestra.run.smithi150.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 740 kB/s | 80 kB 00:00 2024-04-25T11:54:21.867 INFO:teuthology.orchestra.run.smithi150.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-25T11:54:21.917 INFO:teuthology.orchestra.run.smithi150.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-25T11:54:21.968 INFO:teuthology.orchestra.run.smithi150.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 803 kB 00:00 2024-04-25T11:54:22.027 INFO:teuthology.orchestra.run.smithi150.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-25T11:54:22.052 INFO:teuthology.orchestra.run.smithi150.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-25T11:54:22.077 INFO:teuthology.orchestra.run.smithi150.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T11:54:22.103 INFO:teuthology.orchestra.run.smithi150.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T11:54:22.128 INFO:teuthology.orchestra.run.smithi150.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T11:54:22.479 INFO:teuthology.orchestra.run.smithi019.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 2.2 MB/s | 2.4 MB 00:01 2024-04-25T11:54:22.536 INFO:teuthology.orchestra.run.smithi169.stdout:(8/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 4.6 MB/s | 17 MB 00:03 2024-04-25T11:54:22.636 INFO:teuthology.orchestra.run.smithi169.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 253 kB/s | 25 kB 00:00 2024-04-25T11:54:22.913 INFO:teuthology.orchestra.run.smithi019.stdout:(5/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.8 MB/s | 5.4 MB 00:03 2024-04-25T11:54:22.946 INFO:teuthology.orchestra.run.smithi019.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.2 MB/s | 1.5 MB 00:00 2024-04-25T11:54:23.170 INFO:teuthology.orchestra.run.smithi150.stdout:(48/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 8.6 MB/s | 49 MB 00:05 2024-04-25T11:54:23.195 INFO:teuthology.orchestra.run.smithi150.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 631 kB/s | 689 kB 00:01 2024-04-25T11:54:23.229 INFO:teuthology.orchestra.run.smithi150.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 857 kB/s | 943 kB 00:01 2024-04-25T11:54:23.353 INFO:teuthology.orchestra.run.smithi169.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 3.4 MB/s | 11 MB 00:03 2024-04-25T11:54:23.462 INFO:teuthology.orchestra.run.smithi150.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 81 kB/s | 21 kB 00:00 2024-04-25T11:54:23.579 INFO:teuthology.orchestra.run.smithi150.stdout:(52/152): boost-program-options-1.75.0-8.el9.x8 261 kB/s | 107 kB 00:00 2024-04-25T11:54:23.653 INFO:teuthology.orchestra.run.smithi169.stdout:(11/152): ceph-common-19.0.0-3156.gd69f13d8.el7 2.8 MB/s | 21 MB 00:07 2024-04-25T11:54:23.678 INFO:teuthology.orchestra.run.smithi169.stdout:(12/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 691 kB/s | 225 kB 00:00 2024-04-25T11:54:23.720 INFO:teuthology.orchestra.run.smithi169.stdout:(13/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 406 kB/s | 27 kB 00:00 2024-04-25T11:54:23.721 INFO:teuthology.orchestra.run.smithi150.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 637 kB/s | 313 kB 00:00 2024-04-25T11:54:23.746 INFO:teuthology.orchestra.run.smithi150.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 115 kB/s | 33 kB 00:00 2024-04-25T11:54:23.770 INFO:teuthology.orchestra.run.smithi169.stdout:(14/152): libcephfs-devel-19.0.0-3156.gd69f13d8 345 kB/s | 32 kB 00:00 2024-04-25T11:54:23.904 INFO:teuthology.orchestra.run.smithi169.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.2 MB/s | 163 kB 00:00 2024-04-25T11:54:23.938 INFO:teuthology.orchestra.run.smithi150.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 58 kB/s | 11 kB 00:00 2024-04-25T11:54:23.954 INFO:teuthology.orchestra.run.smithi169.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 2.5 MB/s | 126 kB 00:00 2024-04-25T11:54:23.980 INFO:teuthology.orchestra.run.smithi150.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 70 kB/s | 18 kB 00:00 2024-04-25T11:54:23.981 INFO:teuthology.orchestra.run.smithi169.stdout:(17/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 2.8 MB/s | 747 kB 00:00 2024-04-25T11:54:24.065 INFO:teuthology.orchestra.run.smithi169.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 4.4 MB/s | 500 kB 00:00 2024-04-25T11:54:24.124 INFO:teuthology.orchestra.run.smithi169.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 769 kB/s | 45 kB 00:00 2024-04-25T11:54:24.183 INFO:teuthology.orchestra.run.smithi169.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 2.3 MB/s | 135 kB 00:00 2024-04-25T11:54:24.200 INFO:teuthology.orchestra.run.smithi019.stdout:(7/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 4.9 MB/s | 21 MB 00:04 2024-04-25T11:54:24.242 INFO:teuthology.orchestra.run.smithi169.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.7 MB/s | 161 kB 00:00 2024-04-25T11:54:24.314 INFO:teuthology.orchestra.run.smithi150.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 136 kB/s | 45 kB 00:00 2024-04-25T11:54:24.400 INFO:teuthology.orchestra.run.smithi169.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.0 MB/s | 320 kB 00:00 2024-04-25T11:54:24.439 INFO:teuthology.orchestra.run.smithi150.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 320 kB/s | 160 kB 00:00 2024-04-25T11:54:24.584 INFO:teuthology.orchestra.run.smithi169.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.6 MB/s | 298 kB 00:00 2024-04-25T11:54:24.692 INFO:teuthology.orchestra.run.smithi169.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 919 kB/s | 99 kB 00:00 2024-04-25T11:54:24.776 INFO:teuthology.orchestra.run.smithi169.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.0 MB/s | 85 kB 00:00 2024-04-25T11:54:24.881 INFO:teuthology.orchestra.run.smithi150.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.3 MB/s | 3.0 MB 00:01 2024-04-25T11:54:24.964 INFO:teuthology.orchestra.run.smithi150.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.0 MB/s | 662 kB 00:00 2024-04-25T11:54:25.065 INFO:teuthology.orchestra.run.smithi150.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 931 kB/s | 582 kB 00:00 2024-04-25T11:54:25.157 INFO:teuthology.orchestra.run.smithi150.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 92 kB/s | 25 kB 00:00 2024-04-25T11:54:25.239 INFO:teuthology.orchestra.run.smithi019.stdout:(8/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 7.3 MB/s | 17 MB 00:02 2024-04-25T11:54:25.314 INFO:teuthology.orchestra.run.smithi019.stdout:(9/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 4.7 MB 00:02 2024-04-25T11:54:25.323 INFO:teuthology.orchestra.run.smithi150.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 132 kB/s | 47 kB 00:00 2024-04-25T11:54:25.339 INFO:teuthology.orchestra.run.smithi019.stdout:(10/152): ceph-selinux-19.0.0-3156.gd69f13d8.el 252 kB/s | 25 kB 00:00 2024-04-25T11:54:25.460 INFO:teuthology.orchestra.run.smithi169.stdout:(26/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 3.6 MB/s | 5.4 MB 00:01 2024-04-25T11:54:25.465 INFO:teuthology.orchestra.run.smithi019.stdout:(11/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 8.4 MB/s | 11 MB 00:01 2024-04-25T11:54:25.490 INFO:teuthology.orchestra.run.smithi019.stdout:(12/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.5 MB/s | 225 kB 00:00 2024-04-25T11:54:25.518 INFO:teuthology.orchestra.run.smithi169.stdout:(27/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 4.2 MB/s | 3.1 MB 00:00 2024-04-25T11:54:25.523 INFO:teuthology.orchestra.run.smithi019.stdout:(13/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 463 kB/s | 27 kB 00:00 2024-04-25T11:54:25.543 INFO:teuthology.orchestra.run.smithi169.stdout:(28/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 170 kB 00:00 2024-04-25T11:54:25.549 INFO:teuthology.orchestra.run.smithi019.stdout:(14/152): libcephfs-devel-19.0.0-3156.gd69f13d8 542 kB/s | 32 kB 00:00 2024-04-25T11:54:25.569 INFO:teuthology.orchestra.run.smithi169.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 682 kB/s | 34 kB 00:00 2024-04-25T11:54:25.599 INFO:teuthology.orchestra.run.smithi150.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 128 kB/s | 35 kB 00:00 2024-04-25T11:54:25.603 INFO:teuthology.orchestra.run.smithi169.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.4 MB/s | 142 kB 00:00 2024-04-25T11:54:25.616 INFO:teuthology.orchestra.run.smithi019.stdout:(15/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 7.9 MB/s | 747 kB 00:00 2024-04-25T11:54:25.642 INFO:teuthology.orchestra.run.smithi019.stdout:(16/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.7 MB/s | 163 kB 00:00 2024-04-25T11:54:25.650 INFO:teuthology.orchestra.run.smithi150.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 383 kB/s | 188 kB 00:00 2024-04-25T11:54:25.685 INFO:teuthology.orchestra.run.smithi019.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 1.8 MB/s | 126 kB 00:00 2024-04-25T11:54:25.734 INFO:teuthology.orchestra.run.smithi019.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 5.3 MB/s | 500 kB 00:00 2024-04-25T11:54:25.808 INFO:teuthology.orchestra.run.smithi019.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 603 kB/s | 45 kB 00:00 2024-04-25T11:54:25.901 INFO:teuthology.orchestra.run.smithi019.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.4 MB/s | 135 kB 00:00 2024-04-25T11:54:25.908 INFO:teuthology.orchestra.run.smithi150.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 347 kB/s | 292 kB 00:00 2024-04-25T11:54:25.993 INFO:teuthology.orchestra.run.smithi019.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.7 MB/s | 161 kB 00:00 2024-04-25T11:54:26.101 INFO:teuthology.orchestra.run.smithi019.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.9 MB/s | 320 kB 00:00 2024-04-25T11:54:26.260 INFO:teuthology.orchestra.run.smithi019.stdout:(23/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 9.3 MB/s | 5.4 MB 00:00 2024-04-25T11:54:26.285 INFO:teuthology.orchestra.run.smithi019.stdout:(24/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.6 MB/s | 298 kB 00:00 2024-04-25T11:54:26.303 INFO:teuthology.orchestra.run.smithi169.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 4.8 MB/s | 3.5 MB 00:00 2024-04-25T11:54:26.336 INFO:teuthology.orchestra.run.smithi019.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 99 kB 00:00 2024-04-25T11:54:26.361 INFO:teuthology.orchestra.run.smithi019.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T11:54:26.369 INFO:teuthology.orchestra.run.smithi169.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 3.7 MB/s | 249 kB 00:00 2024-04-25T11:54:26.430 INFO:teuthology.orchestra.run.smithi169.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 845 kB/s | 49 kB 00:00 2024-04-25T11:54:26.436 INFO:teuthology.orchestra.run.smithi019.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.2 MB/s | 170 kB 00:00 2024-04-25T11:54:26.478 INFO:teuthology.orchestra.run.smithi169.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 329 kB/s | 16 kB 00:00 2024-04-25T11:54:26.520 INFO:teuthology.orchestra.run.smithi019.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 410 kB/s | 34 kB 00:00 2024-04-25T11:54:26.571 INFO:teuthology.orchestra.run.smithi169.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.9 MB/s | 273 kB 00:00 2024-04-25T11:54:26.621 INFO:teuthology.orchestra.run.smithi019.stdout:(29/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 11 MB/s | 3.1 MB 00:00 2024-04-25T11:54:26.646 INFO:teuthology.orchestra.run.smithi019.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.1 MB/s | 142 kB 00:00 2024-04-25T11:54:26.721 INFO:teuthology.orchestra.run.smithi169.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 5.0 MB/s | 762 kB 00:00 2024-04-25T11:54:26.813 INFO:teuthology.orchestra.run.smithi169.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 199 kB/s | 18 kB 00:00 2024-04-25T11:54:27.079 INFO:teuthology.orchestra.run.smithi019.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 7.7 MB/s | 3.5 MB 00:00 2024-04-25T11:54:27.162 INFO:teuthology.orchestra.run.smithi019.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.9 MB/s | 249 kB 00:00 2024-04-25T11:54:27.188 INFO:teuthology.orchestra.run.smithi169.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 4.7 MB/s | 7.4 MB 00:01 2024-04-25T11:54:27.214 INFO:teuthology.orchestra.run.smithi169.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 201 kB/s | 80 kB 00:00 2024-04-25T11:54:27.247 INFO:teuthology.orchestra.run.smithi019.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 593 kB/s | 49 kB 00:00 2024-04-25T11:54:27.275 INFO:teuthology.orchestra.run.smithi150.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 1.2 MB/s | 1.6 MB 00:01 2024-04-25T11:54:27.338 INFO:teuthology.orchestra.run.smithi019.stdout:(34/152): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-25T11:54:27.363 INFO:teuthology.orchestra.run.smithi019.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 141 kB/s | 16 kB 00:00 2024-04-25T11:54:27.414 INFO:teuthology.orchestra.run.smithi019.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 3.5 MB/s | 273 kB 00:00 2024-04-25T11:54:27.472 INFO:teuthology.orchestra.run.smithi019.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 313 kB/s | 18 kB 00:00 2024-04-25T11:54:27.498 INFO:teuthology.orchestra.run.smithi019.stdout:(38/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 5.5 MB/s | 762 kB 00:00 2024-04-25T11:54:27.575 INFO:teuthology.orchestra.run.smithi150.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 2.4 MB/s | 4.7 MB 00:01 2024-04-25T11:54:27.647 INFO:teuthology.orchestra.run.smithi169.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 451 kB/s | 195 kB 00:00 2024-04-25T11:54:27.650 INFO:teuthology.orchestra.run.smithi150.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 4.4 MB 00:02 2024-04-25T11:54:27.681 INFO:teuthology.orchestra.run.smithi019.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 385 kB/s | 80 kB 00:00 2024-04-25T11:54:27.781 INFO:teuthology.orchestra.run.smithi019.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-04-25T11:54:27.807 INFO:teuthology.orchestra.run.smithi019.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-04-25T11:54:27.841 INFO:teuthology.orchestra.run.smithi019.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-04-25T11:54:27.899 INFO:teuthology.orchestra.run.smithi019.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T11:54:27.922 INFO:teuthology.orchestra.run.smithi169.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-04-25T11:54:27.966 INFO:teuthology.orchestra.run.smithi019.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 7.7 MB/s | 1.2 MB 00:00 2024-04-25T11:54:27.991 INFO:teuthology.orchestra.run.smithi019.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-04-25T11:54:28.000 INFO:teuthology.orchestra.run.smithi150.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 313 kB/s | 133 kB 00:00 2024-04-25T11:54:28.025 INFO:teuthology.orchestra.run.smithi019.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T11:54:28.050 INFO:teuthology.orchestra.run.smithi019.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T11:54:28.126 INFO:teuthology.orchestra.run.smithi150.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 1.0 MB/s | 911 kB 00:00 2024-04-25T11:54:28.147 INFO:teuthology.orchestra.run.smithi169.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 506 kB/s | 253 kB 00:00 2024-04-25T11:54:28.234 INFO:teuthology.orchestra.run.smithi019.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-04-25T11:54:28.276 INFO:teuthology.orchestra.run.smithi019.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-04-25T11:54:28.309 INFO:teuthology.orchestra.run.smithi019.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 943 kB 00:00 2024-04-25T11:54:28.326 INFO:teuthology.orchestra.run.smithi019.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-25T11:54:28.360 INFO:teuthology.orchestra.run.smithi019.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T11:54:28.393 INFO:teuthology.orchestra.run.smithi019.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-04-25T11:54:28.452 INFO:teuthology.orchestra.run.smithi019.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-04-25T11:54:28.502 INFO:teuthology.orchestra.run.smithi019.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-04-25T11:54:28.519 INFO:teuthology.orchestra.run.smithi019.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-04-25T11:54:28.543 INFO:teuthology.orchestra.run.smithi150.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 210 kB/s | 87 kB 00:00 2024-04-25T11:54:28.553 INFO:teuthology.orchestra.run.smithi019.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T11:54:28.578 INFO:teuthology.orchestra.run.smithi019.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-04-25T11:54:28.628 INFO:teuthology.orchestra.run.smithi019.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-04-25T11:54:28.656 INFO:teuthology.orchestra.run.smithi169.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 210 kB/s | 106 kB 00:00 2024-04-25T11:54:28.662 INFO:teuthology.orchestra.run.smithi019.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-04-25T11:54:28.687 INFO:teuthology.orchestra.run.smithi019.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-25T11:54:28.712 INFO:teuthology.orchestra.run.smithi019.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-04-25T11:54:28.738 INFO:teuthology.orchestra.run.smithi019.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T11:54:28.763 INFO:teuthology.orchestra.run.smithi019.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T11:54:28.788 INFO:teuthology.orchestra.run.smithi019.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-04-25T11:54:28.890 INFO:teuthology.orchestra.run.smithi169.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-04-25T11:54:28.905 INFO:teuthology.orchestra.run.smithi019.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-04-25T11:54:28.981 INFO:teuthology.orchestra.run.smithi019.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-25T11:54:29.048 INFO:teuthology.orchestra.run.smithi019.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 13 MB/s | 911 kB 00:00 2024-04-25T11:54:29.074 INFO:teuthology.orchestra.run.smithi169.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 324 kB/s | 135 kB 00:00 2024-04-25T11:54:29.106 INFO:teuthology.orchestra.run.smithi019.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-25T11:54:29.232 INFO:teuthology.orchestra.run.smithi019.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-25T11:54:29.307 INFO:teuthology.orchestra.run.smithi169.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 299 kB/s | 125 kB 00:00 2024-04-25T11:54:29.307 INFO:teuthology.orchestra.run.smithi019.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-04-25T11:54:29.357 INFO:teuthology.orchestra.run.smithi019.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-04-25T11:54:29.382 INFO:teuthology.orchestra.run.smithi019.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-04-25T11:54:29.433 INFO:teuthology.orchestra.run.smithi019.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-25T11:54:29.483 INFO:teuthology.orchestra.run.smithi019.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-04-25T11:54:29.533 INFO:teuthology.orchestra.run.smithi019.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-25T11:54:29.565 INFO:teuthology.orchestra.run.smithi169.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 436 kB/s | 215 kB 00:00 2024-04-25T11:54:29.576 INFO:teuthology.orchestra.run.smithi150.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 3.0 MB/s | 5.8 MB 00:01 2024-04-25T11:54:29.600 INFO:teuthology.orchestra.run.smithi019.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 288 kB/s | 19 kB 00:00 2024-04-25T11:54:29.667 INFO:teuthology.orchestra.run.smithi019.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-25T11:54:29.766 INFO:teuthology.orchestra.run.smithi169.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 689 kB 00:00 2024-04-25T11:54:29.776 INFO:teuthology.orchestra.run.smithi150.stdout:(74/152): protobuf-3.14.0-13.el9.x86_64.rpm 582 kB/s | 1.0 MB 00:01 2024-04-25T11:54:29.784 INFO:teuthology.orchestra.run.smithi019.stdout:(79/152): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-04-25T11:54:29.809 INFO:teuthology.orchestra.run.smithi019.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 245 kB/s | 35 kB 00:00 2024-04-25T11:54:29.858 INFO:teuthology.orchestra.run.smithi169.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-25T11:54:29.860 INFO:teuthology.orchestra.run.smithi019.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-25T11:54:29.891 INFO:teuthology.orchestra.run.smithi169.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-04-25T11:54:29.910 INFO:teuthology.orchestra.run.smithi019.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T11:54:29.925 INFO:teuthology.orchestra.run.smithi169.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 320 kB/s | 21 kB 00:00 2024-04-25T11:54:29.950 INFO:teuthology.orchestra.run.smithi169.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-25T11:54:29.969 INFO:teuthology.orchestra.run.smithi019.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T11:54:29.984 INFO:teuthology.orchestra.run.smithi169.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-04-25T11:54:30.034 INFO:teuthology.orchestra.run.smithi169.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-04-25T11:54:30.052 INFO:teuthology.orchestra.run.smithi019.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-04-25T11:54:30.078 INFO:teuthology.orchestra.run.smithi019.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-04-25T11:54:30.085 INFO:teuthology.orchestra.run.smithi150.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 480 kB/s | 244 kB 00:00 2024-04-25T11:54:30.106 INFO:teuthology.orchestra.run.smithi019.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T11:54:30.126 INFO:teuthology.orchestra.run.smithi169.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-04-25T11:54:30.151 INFO:teuthology.orchestra.run.smithi169.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-04-25T11:54:30.193 INFO:teuthology.orchestra.run.smithi169.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T11:54:30.219 INFO:teuthology.orchestra.run.smithi169.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-04-25T11:54:30.245 INFO:teuthology.orchestra.run.smithi019.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 951 kB/s | 159 kB 00:00 2024-04-25T11:54:30.252 INFO:teuthology.orchestra.run.smithi169.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T11:54:30.270 INFO:teuthology.orchestra.run.smithi019.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-04-25T11:54:30.285 INFO:teuthology.orchestra.run.smithi150.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 487 kB/s | 248 kB 00:00 2024-04-25T11:54:30.287 INFO:teuthology.orchestra.run.smithi019.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-25T11:54:30.294 INFO:teuthology.orchestra.run.smithi169.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 612 kB/s | 25 kB 00:00 2024-04-25T11:54:30.312 INFO:teuthology.orchestra.run.smithi019.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-04-25T11:54:30.352 INFO:teuthology.orchestra.run.smithi150.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 96 kB/s | 26 kB 00:00 2024-04-25T11:54:30.370 INFO:teuthology.orchestra.run.smithi169.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 632 kB/s | 47 kB 00:00 2024-04-25T11:54:30.371 INFO:teuthology.orchestra.run.smithi019.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-04-25T11:54:30.413 INFO:teuthology.orchestra.run.smithi019.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-04-25T11:54:30.428 INFO:teuthology.orchestra.run.smithi169.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-25T11:54:30.463 INFO:teuthology.orchestra.run.smithi019.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T11:54:30.505 INFO:teuthology.orchestra.run.smithi019.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 862 kB/s | 36 kB 00:00 2024-04-25T11:54:30.544 INFO:teuthology.orchestra.run.smithi150.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 74 kB/s | 19 kB 00:00 2024-04-25T11:54:30.552 INFO:teuthology.orchestra.run.smithi169.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-25T11:54:30.579 INFO:teuthology.orchestra.run.smithi169.stdout:(64/152): libtool-2.4.6-45.el9.x86_64.rpm 1.6 MB/s | 582 kB 00:00 2024-04-25T11:54:30.596 INFO:teuthology.orchestra.run.smithi169.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 814 kB/s | 35 kB 00:00 2024-04-25T11:54:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-04-25T11:54:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 246 kB/s | 47 kB 00:00 2024-04-25T11:54:30.739 INFO:teuthology.orchestra.run.smithi019.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-25T11:54:30.765 INFO:teuthology.orchestra.run.smithi019.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 862 kB/s | 58 kB 00:00 2024-04-25T11:54:30.769 INFO:teuthology.orchestra.run.smithi150.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 413 kB/s | 172 kB 00:00 2024-04-25T11:54:30.803 INFO:teuthology.orchestra.run.smithi150.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 134 kB/s | 35 kB 00:00 2024-04-25T11:54:30.807 INFO:teuthology.orchestra.run.smithi019.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-25T11:54:30.813 INFO:teuthology.orchestra.run.smithi169.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-25T11:54:30.832 INFO:teuthology.orchestra.run.smithi019.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 952 kB/s | 64 kB 00:00 2024-04-25T11:54:30.865 INFO:teuthology.orchestra.run.smithi019.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-04-25T11:54:30.907 INFO:teuthology.orchestra.run.smithi019.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-04-25T11:54:30.938 INFO:teuthology.orchestra.run.smithi169.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T11:54:30.958 INFO:teuthology.orchestra.run.smithi019.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 9.2 MB/s | 863 kB 00:00 2024-04-25T11:54:31.005 INFO:teuthology.orchestra.run.smithi169.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 13 MB/s | 911 kB 00:00 2024-04-25T11:54:31.047 INFO:teuthology.orchestra.run.smithi169.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 3.1 MB/s | 133 kB 00:00 2024-04-25T11:54:31.239 INFO:teuthology.orchestra.run.smithi169.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.8 MB 00:00 2024-04-25T11:54:31.258 INFO:teuthology.orchestra.run.smithi019.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-04-25T11:54:31.283 INFO:teuthology.orchestra.run.smithi019.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 950 kB/s | 309 kB 00:00 2024-04-25T11:54:31.306 INFO:teuthology.orchestra.run.smithi169.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T11:54:31.308 INFO:teuthology.orchestra.run.smithi019.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-04-25T11:54:31.311 INFO:teuthology.orchestra.run.smithi150.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 493 kB/s | 250 kB 00:00 2024-04-25T11:54:31.375 INFO:teuthology.orchestra.run.smithi019.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-04-25T11:54:31.425 INFO:teuthology.orchestra.run.smithi019.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 974 kB/s | 49 kB 00:00 2024-04-25T11:54:31.484 INFO:teuthology.orchestra.run.smithi019.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-25T11:54:31.559 INFO:teuthology.orchestra.run.smithi019.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T11:54:31.584 INFO:teuthology.orchestra.run.smithi019.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-25T11:54:31.626 INFO:teuthology.orchestra.run.smithi019.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-04-25T11:54:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-04-25T11:54:31.686 INFO:teuthology.orchestra.run.smithi019.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-04-25T11:54:31.719 INFO:teuthology.orchestra.run.smithi019.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 551 kB/s | 32 kB 00:00 2024-04-25T11:54:31.744 INFO:teuthology.orchestra.run.smithi019.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-04-25T11:54:31.756 INFO:teuthology.orchestra.run.smithi169.stdout:(72/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 5.4 MB/s | 49 MB 00:09 2024-04-25T11:54:31.773 INFO:teuthology.orchestra.run.smithi169.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 187 kB/s | 87 kB 00:00 2024-04-25T11:54:31.786 INFO:teuthology.orchestra.run.smithi019.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-25T11:54:31.811 INFO:teuthology.orchestra.run.smithi019.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-04-25T11:54:31.812 INFO:teuthology.orchestra.run.smithi150.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 444 kB/s | 222 kB 00:00 2024-04-25T11:54:31.840 INFO:teuthology.orchestra.run.smithi169.stdout:(74/152): openblas-openmp-0.3.21-2.el9.x86_64.r 3.8 MB/s | 4.7 MB 00:01 2024-04-25T11:54:31.845 INFO:teuthology.orchestra.run.smithi019.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-04-25T11:54:31.865 INFO:teuthology.orchestra.run.smithi169.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-25T11:54:31.870 INFO:teuthology.orchestra.run.smithi019.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 603 kB/s | 35 kB 00:00 2024-04-25T11:54:31.896 INFO:teuthology.orchestra.run.smithi019.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-25T11:54:31.907 INFO:teuthology.orchestra.run.smithi169.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-25T11:54:31.933 INFO:teuthology.orchestra.run.smithi169.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 384 kB/s | 26 kB 00:00 2024-04-25T11:54:31.958 INFO:teuthology.orchestra.run.smithi169.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-25T11:54:31.962 INFO:teuthology.orchestra.run.smithi019.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 22 MB/s | 2.0 MB 00:00 2024-04-25T11:54:31.983 INFO:teuthology.orchestra.run.smithi169.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-25T11:54:31.988 INFO:teuthology.orchestra.run.smithi019.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-25T11:54:32.008 INFO:teuthology.orchestra.run.smithi169.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-04-25T11:54:32.013 INFO:teuthology.orchestra.run.smithi019.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-25T11:54:32.030 INFO:teuthology.orchestra.run.smithi019.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 421 kB/s | 18 kB 00:00 2024-04-25T11:54:32.055 INFO:teuthology.orchestra.run.smithi019.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 554 kB/s | 23 kB 00:00 2024-04-25T11:54:32.072 INFO:teuthology.orchestra.run.smithi019.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 463 kB/s | 19 kB 00:00 2024-04-25T11:54:32.075 INFO:teuthology.orchestra.run.smithi169.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-25T11:54:32.098 INFO:teuthology.orchestra.run.smithi019.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 466 kB/s | 20 kB 00:00 2024-04-25T11:54:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-25T11:54:32.173 INFO:teuthology.orchestra.run.smithi019.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-25T11:54:32.193 INFO:teuthology.orchestra.run.smithi169.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-04-25T11:54:32.195 INFO:teuthology.orchestra.run.smithi150.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.6 MB/s | 5.1 MB 00:01 2024-04-25T11:54:32.198 INFO:teuthology.orchestra.run.smithi019.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-25T11:54:32.209 INFO:teuthology.orchestra.run.smithi169.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-04-25T11:54:32.224 INFO:teuthology.orchestra.run.smithi019.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-25T11:54:32.229 INFO:teuthology.orchestra.run.smithi150.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 185 kB/s | 77 kB 00:00 2024-04-25T11:54:32.234 INFO:teuthology.orchestra.run.smithi169.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-04-25T11:54:32.257 INFO:teuthology.orchestra.run.smithi019.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-04-25T11:54:32.260 INFO:teuthology.orchestra.run.smithi169.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-25T11:54:32.283 INFO:teuthology.orchestra.run.smithi019.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-25T11:54:32.324 INFO:teuthology.orchestra.run.smithi019.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-25T11:54:32.335 INFO:teuthology.orchestra.run.smithi169.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-25T11:54:32.349 INFO:teuthology.orchestra.run.smithi019.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-04-25T11:54:32.360 INFO:teuthology.orchestra.run.smithi169.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-04-25T11:54:32.375 INFO:teuthology.orchestra.run.smithi019.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-04-25T11:54:32.386 INFO:teuthology.orchestra.run.smithi169.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-04-25T11:54:32.409 INFO:teuthology.orchestra.run.smithi019.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-04-25T11:54:32.411 INFO:teuthology.orchestra.run.smithi169.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T11:54:32.434 INFO:teuthology.orchestra.run.smithi019.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-25T11:54:32.436 INFO:teuthology.orchestra.run.smithi169.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T11:54:32.459 INFO:teuthology.orchestra.run.smithi019.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-04-25T11:54:32.461 INFO:teuthology.orchestra.run.smithi169.stdout:(91/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-04-25T11:54:32.484 INFO:teuthology.orchestra.run.smithi019.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-25T11:54:32.487 INFO:teuthology.orchestra.run.smithi169.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 826 kB/s | 42 kB 00:00 2024-04-25T11:54:32.512 INFO:teuthology.orchestra.run.smithi169.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-04-25T11:54:32.518 INFO:teuthology.orchestra.run.smithi019.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-25T11:54:32.529 INFO:teuthology.orchestra.run.smithi169.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.1 MB/s | 221 kB 00:00 2024-04-25T11:54:32.537 INFO:teuthology.orchestra.run.smithi150.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 124 kB/s | 42 kB 00:00 2024-04-25T11:54:32.543 INFO:teuthology.orchestra.run.smithi019.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-25T11:54:32.563 INFO:teuthology.orchestra.run.smithi169.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-04-25T11:54:32.577 INFO:teuthology.orchestra.run.smithi019.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-04-25T11:54:32.588 INFO:teuthology.orchestra.run.smithi169.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-04-25T11:54:32.602 INFO:teuthology.orchestra.run.smithi019.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-04-25T11:54:32.613 INFO:teuthology.orchestra.run.smithi169.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-25T11:54:32.635 INFO:teuthology.orchestra.run.smithi019.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-25T11:54:32.638 INFO:teuthology.orchestra.run.smithi169.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T11:54:32.655 INFO:teuthology.orchestra.run.smithi150.stdout:(86/152): python3-babel-2.9.1-2.el9.noarch.rpm 1.5 MB/s | 6.0 MB 00:04 2024-04-25T11:54:32.664 INFO:teuthology.orchestra.run.smithi169.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T11:54:32.689 INFO:teuthology.orchestra.run.smithi169.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T11:54:32.703 INFO:teuthology.orchestra.run.smithi019.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-25T11:54:32.723 INFO:teuthology.orchestra.run.smithi169.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 762 kB/s | 45 kB 00:00 2024-04-25T11:54:32.730 INFO:teuthology.orchestra.run.smithi150.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 538 kB/s | 269 kB 00:00 2024-04-25T11:54:32.907 INFO:teuthology.orchestra.run.smithi169.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-04-25T11:54:32.997 INFO:teuthology.orchestra.run.smithi150.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 346 kB/s | 159 kB 00:00 2024-04-25T11:54:33.064 INFO:teuthology.orchestra.run.smithi150.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 153 kB/s | 51 kB 00:00 2024-04-25T11:54:33.086 INFO:teuthology.orchestra.run.smithi019.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 7.5 MB/s | 3.4 MB 00:00 2024-04-25T11:54:33.182 INFO:teuthology.orchestra.run.smithi169.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 45 kB/s | 22 kB 00:00 2024-04-25T11:54:33.203 INFO:teuthology.orchestra.run.smithi019.stdout:(149/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 6.3 MB/s | 3.1 MB 00:00 2024-04-25T11:54:33.306 INFO:teuthology.orchestra.run.smithi150.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 428 kB/s | 279 kB 00:00 2024-04-25T11:54:33.331 INFO:teuthology.orchestra.run.smithi150.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 161 kB/s | 54 kB 00:00 2024-04-25T11:54:33.353 INFO:teuthology.orchestra.run.smithi019.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 5.1 MB/s | 1.3 MB 00:00 2024-04-25T11:54:33.399 INFO:teuthology.orchestra.run.smithi169.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-04-25T11:54:33.537 INFO:teuthology.orchestra.run.smithi019.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.2 MB/s | 2.4 MB 00:00 2024-04-25T11:54:33.573 INFO:teuthology.orchestra.run.smithi150.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 156 kB/s | 42 kB 00:00 2024-04-25T11:54:33.673 INFO:teuthology.orchestra.run.smithi150.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 129 kB/s | 44 kB 00:00 2024-04-25T11:54:33.874 INFO:teuthology.orchestra.run.smithi169.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 43 kB/s | 19 kB 00:00 2024-04-25T11:54:33.940 INFO:teuthology.orchestra.run.smithi150.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 135 kB/s | 36 kB 00:00 2024-04-25T11:54:34.049 INFO:teuthology.orchestra.run.smithi169.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 480 kB/s | 548 kB 00:01 2024-04-25T11:54:34.166 INFO:teuthology.orchestra.run.smithi169.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 223 kB/s | 26 kB 00:00 2024-04-25T11:54:34.248 INFO:teuthology.orchestra.run.smithi150.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 327 kB/s | 221 kB 00:00 2024-04-25T11:54:34.250 INFO:teuthology.orchestra.run.smithi169.stdout:(108/152): gperftools-libs-2.9.1-2.el9.x86_64.r 289 kB/s | 309 kB 00:01 2024-04-25T11:54:34.266 INFO:teuthology.orchestra.run.smithi169.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2024-04-25T11:54:34.274 INFO:teuthology.orchestra.run.smithi150.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 142 kB/s | 47 kB 00:00 2024-04-25T11:54:34.434 INFO:teuthology.orchestra.run.smithi169.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2024-04-25T11:54:34.542 INFO:teuthology.orchestra.run.smithi169.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 548 kB/s | 151 kB 00:00 2024-04-25T11:54:34.604 INFO:teuthology.orchestra.run.smithi019.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 5.3 MB/s | 49 MB 00:09 2024-04-25T11:54:34.608 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:54:34.608 INFO:teuthology.orchestra.run.smithi019.stdout:Total 14 MB/s | 222 MB 00:15 2024-04-25T11:54:34.616 INFO:teuthology.orchestra.run.smithi150.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 169 kB/s | 58 kB 00:00 2024-04-25T11:54:34.766 INFO:teuthology.orchestra.run.smithi150.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 589 kB/s | 305 kB 00:00 2024-04-25T11:54:35.025 INFO:teuthology.orchestra.run.smithi150.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 229 kB/s | 94 kB 00:00 2024-04-25T11:54:35.117 INFO:teuthology.orchestra.run.smithi150.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 182 kB/s | 64 kB 00:00 2024-04-25T11:54:35.227 INFO:teuthology.orchestra.run.smithi150.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-04-25T11:54:35.376 INFO:teuthology.orchestra.run.smithi150.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 128 kB/s | 45 kB 00:00 2024-04-25T11:54:35.401 INFO:teuthology.orchestra.run.smithi169.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 639 kB/s | 548 kB 00:00 2024-04-25T11:54:35.518 INFO:teuthology.orchestra.run.smithi169.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 373 kB/s | 43 kB 00:00 2024-04-25T11:54:35.543 INFO:teuthology.orchestra.run.smithi150.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-04-25T11:54:35.755 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-04-25T11:54:35.768 INFO:teuthology.orchestra.run.smithi169.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 129 kB/s | 32 kB 00:00 2024-04-25T11:54:35.802 INFO:teuthology.orchestra.run.smithi169.stdout:(115/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 613 kB/s | 838 kB 00:01 2024-04-25T11:54:35.877 INFO:teuthology.orchestra.run.smithi169.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-25T11:54:35.914 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-04-25T11:54:35.914 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-04-25T11:54:35.976 INFO:teuthology.orchestra.run.smithi150.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 913 kB/s | 548 kB 00:00 2024-04-25T11:54:36.052 INFO:teuthology.orchestra.run.smithi150.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 260 kB/s | 19 kB 00:00 2024-04-25T11:54:36.077 INFO:teuthology.orchestra.run.smithi150.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 579 kB/s | 309 kB 00:00 2024-04-25T11:54:36.094 INFO:teuthology.orchestra.run.smithi169.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 588 kB/s | 172 kB 00:00 2024-04-25T11:54:36.152 INFO:teuthology.orchestra.run.smithi150.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-04-25T11:54:36.227 INFO:teuthology.orchestra.run.smithi150.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 646 kB/s | 49 kB 00:00 2024-04-25T11:54:36.311 INFO:teuthology.orchestra.run.smithi150.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 810 kB/s | 67 kB 00:00 2024-04-25T11:54:36.370 INFO:teuthology.orchestra.run.smithi150.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-25T11:54:36.378 INFO:teuthology.orchestra.run.smithi169.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 717 kB/s | 358 kB 00:00 2024-04-25T11:54:36.403 INFO:teuthology.orchestra.run.smithi150.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-25T11:54:36.462 INFO:teuthology.orchestra.run.smithi150.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-04-25T11:54:36.478 INFO:teuthology.orchestra.run.smithi169.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 353 kB/s | 35 kB 00:00 2024-04-25T11:54:36.520 INFO:teuthology.orchestra.run.smithi169.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 712 kB/s | 303 kB 00:00 2024-04-25T11:54:36.529 INFO:teuthology.orchestra.run.smithi150.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 653 kB/s | 43 kB 00:00 2024-04-25T11:54:36.571 INFO:teuthology.orchestra.run.smithi150.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-04-25T11:54:36.604 INFO:teuthology.orchestra.run.smithi150.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-04-25T11:54:36.638 INFO:teuthology.orchestra.run.smithi150.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-25T11:54:36.680 INFO:teuthology.orchestra.run.smithi150.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-25T11:54:36.730 INFO:teuthology.orchestra.run.smithi150.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-25T11:54:36.755 INFO:teuthology.orchestra.run.smithi150.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-25T11:54:36.820 INFO:teuthology.orchestra.run.smithi169.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 601 kB/s | 206 kB 00:00 2024-04-25T11:54:36.922 INFO:teuthology.orchestra.run.smithi150.stdout:(120/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.2 MB/s | 16 MB 00:03 2024-04-25T11:54:36.939 INFO:teuthology.orchestra.run.smithi150.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 169 kB/s | 35 kB 00:00 2024-04-25T11:54:36.964 INFO:teuthology.orchestra.run.smithi150.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 984 kB/s | 206 kB 00:00 2024-04-25T11:54:37.040 INFO:teuthology.orchestra.run.smithi150.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-25T11:54:37.057 INFO:teuthology.orchestra.run.smithi150.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-25T11:54:37.095 INFO:teuthology.orchestra.run.smithi169.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 525 kB/s | 144 kB 00:00 2024-04-25T11:54:37.098 INFO:teuthology.orchestra.run.smithi150.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-04-25T11:54:37.124 INFO:teuthology.orchestra.run.smithi150.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-04-25T11:54:37.166 INFO:teuthology.orchestra.run.smithi150.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 290 kB/s | 19 kB 00:00 2024-04-25T11:54:37.191 INFO:teuthology.orchestra.run.smithi150.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-04-25T11:54:37.195 INFO:teuthology.orchestra.run.smithi169.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-25T11:54:37.260 INFO:teuthology.orchestra.run.smithi150.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-25T11:54:37.285 INFO:teuthology.orchestra.run.smithi150.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 492 kB/s | 46 kB 00:00 2024-04-25T11:54:37.304 INFO:teuthology.orchestra.run.smithi169.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-25T11:54:37.327 INFO:teuthology.orchestra.run.smithi150.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-25T11:54:37.361 INFO:teuthology.orchestra.run.smithi150.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-04-25T11:54:37.403 INFO:teuthology.orchestra.run.smithi150.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-25T11:54:37.405 INFO:teuthology.orchestra.run.smithi169.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-04-25T11:54:37.445 INFO:teuthology.orchestra.run.smithi150.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-25T11:54:37.471 INFO:teuthology.orchestra.run.smithi150.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-25T11:54:37.505 INFO:teuthology.orchestra.run.smithi169.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-04-25T11:54:37.512 INFO:teuthology.orchestra.run.smithi150.stdout:(136/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.5 MB/s | 2.0 MB 00:00 2024-04-25T11:54:37.537 INFO:teuthology.orchestra.run.smithi150.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-04-25T11:54:37.563 INFO:teuthology.orchestra.run.smithi150.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-04-25T11:54:37.588 INFO:teuthology.orchestra.run.smithi150.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-25T11:54:37.605 INFO:teuthology.orchestra.run.smithi169.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-25T11:54:37.613 INFO:teuthology.orchestra.run.smithi150.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 783 kB/s | 59 kB 00:00 2024-04-25T11:54:37.639 INFO:teuthology.orchestra.run.smithi150.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-04-25T11:54:37.656 INFO:teuthology.orchestra.run.smithi150.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 757 kB/s | 51 kB 00:00 2024-04-25T11:54:37.697 INFO:teuthology.orchestra.run.smithi150.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-25T11:54:37.723 INFO:teuthology.orchestra.run.smithi150.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-25T11:54:37.748 INFO:teuthology.orchestra.run.smithi150.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-04-25T11:54:37.765 INFO:teuthology.orchestra.run.smithi150.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-04-25T11:54:37.781 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-04-25T11:54:37.782 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-04-25T11:54:37.798 INFO:teuthology.orchestra.run.smithi150.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-25T11:54:37.857 INFO:teuthology.orchestra.run.smithi150.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-25T11:54:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:(149/152): librados2-19.0.0-3156.gd69f13d8.el7. 10 MB/s | 3.4 MB 00:00 2024-04-25T11:54:38.216 INFO:teuthology.orchestra.run.smithi150.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.8 MB/s | 1.3 MB 00:00 2024-04-25T11:54:38.374 INFO:teuthology.orchestra.run.smithi150.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.5 MB/s | 2.4 MB 00:00 2024-04-25T11:54:38.600 INFO:teuthology.orchestra.run.smithi150.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 3.9 MB/s | 3.1 MB 00:00 2024-04-25T11:54:38.605 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:54:38.606 INFO:teuthology.orchestra.run.smithi150.stdout:Total 9.0 MB/s | 222 MB 00:24 2024-04-25T11:54:39.115 INFO:teuthology.orchestra.run.smithi169.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 693 kB/s | 1.0 MB 00:01 2024-04-25T11:54:39.241 INFO:teuthology.orchestra.run.smithi169.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 767 kB/s | 2.0 MB 00:02 2024-04-25T11:54:39.290 INFO:teuthology.orchestra.run.smithi169.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 264 kB/s | 46 kB 00:00 2024-04-25T11:54:39.431 INFO:teuthology.orchestra.run.smithi169.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 415 kB/s | 79 kB 00:00 2024-04-25T11:54:39.473 INFO:teuthology.orchestra.run.smithi169.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 315 kB/s | 58 kB 00:00 2024-04-25T11:54:39.574 INFO:teuthology.orchestra.run.smithi169.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-25T11:54:39.724 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-25T11:54:39.757 INFO:teuthology.orchestra.run.smithi169.stdout:(134/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 491 kB/s | 90 kB 00:00 2024-04-25T11:54:39.858 INFO:teuthology.orchestra.run.smithi169.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-04-25T11:54:39.872 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-25T11:54:39.873 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-25T11:54:39.883 INFO:teuthology.orchestra.run.smithi169.stdout:(136/152): python3-pecan-1.4.2-3.el9.noarch.rpm 604 kB/s | 272 kB 00:00 2024-04-25T11:54:39.942 INFO:teuthology.orchestra.run.smithi169.stdout:(137/152): libarrow-9.0.0-10.el9.x86_64.rpm 746 kB/s | 4.4 MB 00:06 2024-04-25T11:54:39.967 INFO:teuthology.orchestra.run.smithi169.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2024-04-25T11:54:40.067 INFO:teuthology.orchestra.run.smithi169.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 357 kB/s | 36 kB 00:00 2024-04-25T11:54:40.125 INFO:teuthology.orchestra.run.smithi169.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 322 kB/s | 59 kB 00:00 2024-04-25T11:54:40.176 INFO:teuthology.orchestra.run.smithi169.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 644 kB/s | 188 kB 00:00 2024-04-25T11:54:40.251 INFO:teuthology.orchestra.run.smithi169.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 277 kB/s | 51 kB 00:00 2024-04-25T11:54:40.360 INFO:teuthology.orchestra.run.smithi169.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 488 kB/s | 90 kB 00:00 2024-04-25T11:54:40.468 INFO:teuthology.orchestra.run.smithi169.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-04-25T11:54:40.493 INFO:teuthology.orchestra.run.smithi169.stdout:(145/152): python3-webob-1.8.7-6.el9.noarch.rpm 624 kB/s | 230 kB 00:00 2024-04-25T11:54:40.829 INFO:teuthology.orchestra.run.smithi169.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 532 kB/s | 192 kB 00:00 2024-04-25T11:54:40.877 INFO:teuthology.orchestra.run.smithi169.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 682 kB/s | 427 kB 00:00 2024-04-25T11:54:41.294 INFO:teuthology.orchestra.run.smithi169.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 7.3 MB/s | 3.4 MB 00:00 2024-04-25T11:54:41.486 INFO:teuthology.orchestra.run.smithi169.stdout:(149/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 5.2 MB/s | 3.1 MB 00:00 2024-04-25T11:54:41.672 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-25T11:54:41.672 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-25T11:54:42.170 INFO:teuthology.orchestra.run.smithi169.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 1.5 MB/s | 1.3 MB 00:00 2024-04-25T11:54:42.562 INFO:teuthology.orchestra.run.smithi169.stdout:(151/152): thrift-0.15.0-2.el9.x86_64.rpm 783 kB/s | 1.6 MB 00:02 2024-04-25T11:54:43.791 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-04-25T11:54:43.883 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:54:43.922 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:54:43.984 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:54:44.034 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:54:44.059 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:54:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:54:44.447 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:54:44.453 INFO:teuthology.orchestra.run.smithi169.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 826 kB/s | 2.4 MB 00:02 2024-04-25T11:54:44.462 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:54:44.463 INFO:teuthology.orchestra.run.smithi169.stdout:Total 7.6 MB/s | 222 MB 00:29 2024-04-25T11:54:44.560 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:54:44.684 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:54:44.711 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:54:44.802 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:54:44.832 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:54:44.988 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:54:45.037 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:54:45.065 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:54:45.153 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:54:45.183 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:54:45.245 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:54:45.267 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:54:45.289 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:54:45.484 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:54:45.516 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:54:45.532 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-04-25T11:54:45.628 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:54:45.677 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-04-25T11:54:45.677 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-04-25T11:54:45.733 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:54:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:54:45.809 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:54:45.885 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:54:45.925 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:54:45.969 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:54:46.001 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:54:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:54:46.101 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:54:46.161 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:54:46.190 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:54:46.234 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:54:46.312 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:54:46.344 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T11:54:46.388 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T11:54:46.416 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T11:54:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:54:46.556 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:54:46.632 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:54:46.769 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:54:46.825 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:54:47.377 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:54:47.462 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-04-25T11:54:47.462 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-04-25T11:54:47.463 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:54:47.499 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:54:47.542 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:54:47.580 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T11:54:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-25T11:54:47.622 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:54:47.704 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:54:47.778 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:54:47.834 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:54:47.841 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:54:47.888 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:54:47.897 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:54:47.930 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:54:47.955 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:54:47.970 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:54:47.998 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:54:48.000 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:54:48.037 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:54:48.084 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:54:48.119 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:54:48.163 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:54:48.205 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:54:48.326 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:54:48.346 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:54:48.431 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:54:48.435 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:54:48.465 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:54:48.507 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:54:48.545 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:54:48.574 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:54:48.604 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:54:48.640 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:54:48.665 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:54:48.680 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:54:48.695 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:54:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:54:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:54:48.807 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:54:48.860 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:54:48.918 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:54:48.945 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:54:48.945 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:54:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:54:49.007 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:54:49.041 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:54:49.045 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:54:49.071 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:54:49.126 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:54:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:54:49.175 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:54:49.189 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:54:49.321 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:54:49.331 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:54:49.350 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:54:49.364 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:54:49.385 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:54:49.418 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:54:49.461 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:54:49.474 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:54:49.511 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:54:49.569 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:54:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:54:49.613 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:54:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:54:49.650 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:54:49.681 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:54:49.721 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:54:49.729 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:54:49.755 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:54:49.799 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:54:49.831 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:54:49.841 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:54:49.859 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:54:49.899 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:54:49.928 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:54:49.990 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:54:50.020 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:54:50.055 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:54:50.132 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:54:50.166 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T11:54:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T11:54:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T11:54:50.319 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:54:50.361 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:54:50.436 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:54:50.578 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:54:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:54:50.654 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:54:50.695 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:54:50.729 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:54:50.831 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:54:50.888 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:54:50.978 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:54:51.056 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:54:51.153 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:54:51.234 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:54:51.240 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:54:51.263 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:54:51.278 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:54:51.306 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:54:51.344 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T11:54:51.358 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:54:51.386 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:54:51.551 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:54:51.606 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:54:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:54:51.734 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:54:51.765 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:54:51.802 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:54:51.848 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:54:51.884 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:54:51.927 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:54:51.994 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:54:52.033 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:54:52.131 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:54:52.155 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:54:52.174 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:54:52.204 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:54:52.246 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:54:52.341 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:54:52.371 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:54:52.411 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:54:52.437 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:54:52.461 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:54:52.537 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:54:52.663 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:54:52.693 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:54:52.721 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:54:52.766 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:54:52.804 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:54:52.834 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:54:52.864 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:54:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:54:52.932 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:54:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:54:53.009 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:54:53.099 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:54:53.147 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:54:53.187 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:54:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:54:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:54:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:54:53.404 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-04-25T11:54:53.445 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:54:53.499 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:54:53.515 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:54:53.538 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:54:53.543 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:54:53.576 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:54:53.600 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:54:53.642 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:54:53.667 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:54:53.710 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:54:53.904 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:54:53.947 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:54:54.030 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:54:54.047 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:54:54.061 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T11:54:54.097 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:54:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:54:54.151 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:54:54.169 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:54:54.202 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:54:54.265 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:54:54.302 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:54:54.319 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:54:54.360 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:54:54.436 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:54:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:54:54.473 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:54:54.488 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:54:54.601 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:54:54.624 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:54:54.658 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:54:54.667 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:54:54.678 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:54:54.685 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:54:54.707 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:54:54.730 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:54:54.752 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T11:54:54.764 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:54:54.810 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:54:54.833 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:54:54.895 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:54:54.917 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:54:54.937 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:54:55.134 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:54:55.166 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:54:55.274 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:54:55.376 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:54:55.407 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:54:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:54:55.450 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:54:55.515 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:54:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:54:55.549 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:54:55.593 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:54:55.626 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:54:55.653 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:54:55.722 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:54:55.776 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:54:55.807 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:54:55.841 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:54:55.918 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:54:55.952 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T11:54:56.004 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T11:54:56.033 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T11:54:56.102 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:54:56.148 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:54:56.222 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:54:56.363 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:54:56.416 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:54:56.763 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:54:56.864 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:54:56.893 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:54:56.926 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:54:56.942 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:54:57.029 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:54:57.067 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:54:57.101 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:54:57.118 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:54:57.140 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:54:57.156 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T11:54:57.198 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:54:57.239 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:54:57.280 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:54:57.360 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:54:57.361 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:54:57.393 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T11:54:57.408 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:54:57.429 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:54:57.464 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:54:57.536 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:54:57.568 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:54:57.604 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:54:57.651 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:54:57.687 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:54:57.709 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:54:57.730 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:54:57.743 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:54:57.772 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:54:57.834 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:54:57.834 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:54:57.835 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:54:57.835 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:54:57.861 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:54:57.908 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:54:57.910 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:54:57.935 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:54:57.970 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:54:58.000 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:54:58.004 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:54:58.035 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T11:54:58.041 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:54:58.139 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:54:58.183 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:54:58.223 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:54:58.249 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:54:58.274 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:54:58.349 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:54:58.476 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:54:58.514 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:54:58.543 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:54:58.577 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:54:58.616 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:54:58.648 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:54:58.694 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:54:58.729 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:54:58.762 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:54:58.804 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:54:58.838 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:54:58.925 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:54:58.977 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:54:59.008 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:54:59.055 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:54:59.155 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:54:59.210 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:54:59.936 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:54:59.982 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:55:00.016 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:55:00.116 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:55:00.190 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:55:00.277 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:55:00.309 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:55:00.318 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:55:00.339 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:55:00.497 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:55:00.524 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:55:00.603 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:55:00.882 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:55:00.927 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:55:01.021 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:55:01.021 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:55:01.021 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:55:01.021 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:55:01.052 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:55:01.245 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:55:01.364 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:55:02.586 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:55:02.662 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:55:02.689 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:55:02.723 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:55:03.038 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:55:03.085 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:55:03.176 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:55:03.207 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T11:55:03.243 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:55:03.725 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:55:03.763 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:55:03.843 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:55:03.865 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T11:55:06.159 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:55:06.195 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:55:06.738 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:55:06.782 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:55:06.853 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:55:06.853 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:55:06.854 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:55:06.854 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:55:06.883 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:55:09.693 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:55:09.693 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-04-25T11:55:09.693 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-04-25T11:55:09.693 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-04-25T11:55:09.693 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-04-25T11:55:09.693 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-04-25T11:55:09.693 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-04-25T11:55:09.693 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-04-25T11:55:09.693 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:55:09.873 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:55:09.889 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:55:09.889 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:09.889 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:55:09.889 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:55:11.418 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:55:11.447 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:55:11.447 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:11.447 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:55:11.447 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:55:11.447 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:55:11.448 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:55:11.677 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:55:11.700 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:55:11.700 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:11.701 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:55:11.701 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:55:11.701 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:55:11.701 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:55:12.168 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:55:12.192 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:55:12.193 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:12.193 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:55:12.193 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:55:12.193 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:55:12.193 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:55:12.299 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:55:12.357 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:55:12.421 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:55:12.622 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:55:12.670 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:55:12.716 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:55:12.716 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-04-25T11:55:12.716 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-04-25T11:55:12.716 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-04-25T11:55:12.716 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-04-25T11:55:12.716 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-04-25T11:55:12.716 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-04-25T11:55:12.716 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-04-25T11:55:12.716 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:55:12.750 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:55:12.848 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:55:12.898 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:55:12.914 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:55:12.914 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:12.915 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:55:12.915 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:55:12.988 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:55:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:55:13.060 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:55:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:55:13.548 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:55:13.577 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:55:13.610 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T11:55:13.673 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T11:55:13.708 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:55:14.402 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:55:14.427 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:55:14.427 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:14.428 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:55:14.428 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:55:14.428 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:55:14.428 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:55:14.652 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:55:14.675 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:55:14.675 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:14.675 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:55:14.675 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:55:14.675 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:55:14.675 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:55:14.801 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:55:14.835 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:55:15.111 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:55:15.134 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:55:15.138 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:15.138 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:55:15.138 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:55:15.138 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:55:15.138 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:55:15.241 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:55:15.290 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:55:15.346 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:55:15.536 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:55:15.595 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:55:15.673 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:55:15.761 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:55:15.848 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:55:15.869 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:55:15.901 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:55:15.937 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:55:15.969 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:55:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:55:15.997 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:55:16.156 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T11:55:16.187 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:55:16.212 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:55:16.213 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:16.213 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:55:16.213 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:55:16.213 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:55:16.213 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:55:16.263 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:55:16.275 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:55:16.376 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T11:55:16.429 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T11:55:16.432 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:55:16.462 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:55:16.494 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T11:55:16.555 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T11:55:16.593 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:55:17.403 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:55:17.441 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:55:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:55:17.465 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:17.465 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:55:17.465 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:55:17.465 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:55:17.465 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:55:17.508 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:55:17.532 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:55:17.532 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:17.532 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:55:17.532 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:55:17.583 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:55:17.607 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:55:17.607 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:17.607 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:55:17.608 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:55:17.608 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:55:17.608 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:55:17.637 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:55:17.678 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:55:17.926 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:55:17.950 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:55:17.950 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:17.950 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:55:17.950 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:55:17.951 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:55:17.951 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:55:18.190 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:55:18.190 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2024-04-25T11:55:18.190 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2024-04-25T11:55:18.191 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2024-04-25T11:55:18.191 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2024-04-25T11:55:18.191 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2024-04-25T11:55:18.191 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2024-04-25T11:55:18.191 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2024-04-25T11:55:18.191 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:55:18.379 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:55:18.395 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:55:18.395 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:18.395 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:55:18.395 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:55:18.641 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:55:18.662 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:55:18.766 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:55:18.943 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T11:55:18.972 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:55:18.996 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:55:18.996 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:18.996 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:55:18.996 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:55:18.996 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:55:18.996 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:55:19.040 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:55:19.052 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:55:19.149 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T11:55:19.196 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T11:55:19.883 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:55:19.908 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:55:19.908 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:19.908 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:55:19.908 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:55:19.909 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:55:19.909 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:55:20.117 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:55:20.125 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:55:20.147 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:55:20.147 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:20.147 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:55:20.147 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:55:20.147 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:55:20.147 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:55:20.150 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:55:20.174 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:55:20.174 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:20.174 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:55:20.174 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:55:20.174 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:55:20.174 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:55:20.218 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:55:20.240 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:55:20.240 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:20.240 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:55:20.240 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:55:20.285 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:55:20.307 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:55:20.308 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:20.308 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:55:20.308 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:55:20.308 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:55:20.308 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:55:20.584 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:55:20.606 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:55:20.606 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:20.606 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:55:20.606 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:55:20.606 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:55:20.606 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:55:20.628 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:55:20.650 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:55:20.651 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:20.651 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:55:20.651 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:55:20.651 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:55:20.651 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:55:20.706 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:55:20.763 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:55:20.817 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:55:21.004 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:55:21.060 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:55:21.137 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:55:21.226 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:55:21.382 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:55:21.426 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:55:21.458 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:55:21.486 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:55:21.912 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:55:21.935 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:55:21.958 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T11:55:22.020 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T11:55:22.049 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:55:22.796 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T11:55:22.838 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T11:55:22.878 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T11:55:22.912 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:55:22.939 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:55:23.086 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:55:23.126 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:55:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T11:55:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:55:23.207 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T11:55:23.876 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:55:23.913 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:55:23.934 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:55:24.089 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:55:24.118 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:55:24.219 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:55:24.400 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T11:55:24.428 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:55:24.451 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:55:24.451 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:24.451 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:55:24.451 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:55:24.451 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:55:24.451 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:55:24.496 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:55:24.506 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:55:24.605 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T11:55:24.652 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T11:55:24.835 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T11:55:24.977 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:55:25.027 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:55:25.112 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:55:25.148 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:55:25.148 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:55:25.232 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:55:25.248 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T11:55:25.264 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:55:25.264 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:55:25.289 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T11:55:25.330 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T11:55:25.351 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:55:25.362 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:55:25.389 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:55:25.424 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:55:25.572 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:55:25.573 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T11:55:25.607 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:55:25.619 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:55:25.630 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:55:25.630 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:25.630 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:55:25.630 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:55:25.630 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:55:25.630 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:55:25.640 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T11:55:25.674 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:55:25.696 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:55:25.696 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:25.696 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:55:25.696 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:55:25.742 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:55:25.764 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:55:25.764 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:25.764 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:55:25.764 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:55:25.764 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:55:25.764 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:55:26.067 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:55:26.090 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:55:26.090 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:55:26.090 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:55:26.090 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:55:26.090 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:55:26.091 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:55:26.200 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:55:26.238 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:55:26.258 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:55:27.117 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T11:55:27.193 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:55:27.234 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:55:27.316 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:55:27.355 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:55:27.355 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:55:27.440 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:55:27.472 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:55:27.472 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:55:27.567 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:55:27.637 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:55:30.679 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T11:55:30.720 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T11:55:30.762 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T11:55:30.793 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:55:30.821 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:55:30.996 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T11:55:30.999 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:55:30.999 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T11:55:30.999 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T11:55:30.999 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T11:55:30.999 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T11:55:30.999 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T11:55:30.999 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T11:55:30.999 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T11:55:30.999 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:55:30.999 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:55:30.999 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:55:30.999 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T11:55:31.000 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T11:55:31.000 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:55:31.000 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T11:55:31.000 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T11:55:31.000 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T11:55:31.000 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T11:55:31.000 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T11:55:31.000 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T11:55:31.000 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T11:55:31.000 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T11:55:31.000 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T11:55:31.000 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T11:55:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T11:55:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T11:55:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T11:55:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T11:55:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T11:55:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T11:55:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T11:55:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T11:55:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T11:55:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T11:55:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T11:55:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T11:55:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T11:55:31.001 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T11:55:31.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T11:55:31.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T11:55:31.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:55:31.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:55:31.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:55:31.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:55:31.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:55:31.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:55:31.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:55:31.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:55:31.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:55:31.002 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:55:31.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:55:31.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:55:31.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:55:31.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:55:31.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:55:31.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:55:31.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:55:31.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:55:31.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:55:31.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:55:31.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:55:31.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:55:31.003 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:55:31.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:55:31.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:55:31.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:55:31.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:55:31.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:55:31.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:55:31.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:55:31.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:55:31.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:55:31.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:55:31.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:55:31.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:55:31.004 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:55:31.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:55:31.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:55:31.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:55:31.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:55:31.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:55:31.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:55:31.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:55:31.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:55:31.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:55:31.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:55:31.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:55:31.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:55:31.005 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:55:31.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:55:31.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:55:31.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:55:31.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:55:31.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:55:31.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:55:31.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:55:31.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:55:31.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:55:31.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:55:31.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:55:31.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:55:31.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:55:31.006 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:55:31.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:55:31.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:55:31.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:55:31.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:55:31.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:55:31.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:55:31.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:55:31.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:55:31.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:55:31.007 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:55:31.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:55:31.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:55:31.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:55:31.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:55:31.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:55:31.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:55:31.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:55:31.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:55:31.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:55:31.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:55:31.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:55:31.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:55:31.008 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:55:31.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:55:31.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:55:31.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:55:31.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:55:31.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:55:31.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:55:31.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:55:31.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:55:31.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:55:31.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:55:31.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:55:31.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:55:31.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:55:31.009 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:55:31.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:55:31.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:55:31.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:55:31.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:55:31.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:55:31.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:55:31.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:55:31.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:55:31.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:55:31.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T11:55:31.010 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:55:31.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T11:55:31.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:55:31.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:55:31.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:55:31.011 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:55:31.042 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:55:31.072 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T11:55:31.635 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:55:31.677 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:55:31.697 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:55:32.541 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T11:55:32.649 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:55:32.690 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:55:32.771 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:55:32.812 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:55:32.812 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:55:32.896 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:55:32.920 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:55:32.920 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:55:33.010 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:55:33.046 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:55:33.046 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:55:33.046 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-04-25T11:55:33.046 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.046 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.047 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:55:33.047 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:55:33.047 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-04-25T11:55:33.047 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:55:33.047 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:55:33.047 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:55:33.047 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:55:33.047 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.047 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.048 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.048 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.048 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:33.048 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.048 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.048 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.048 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:33.048 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:33.048 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:33.048 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:33.048 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:33.049 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.049 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.049 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:33.049 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.049 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.049 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.049 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:33.049 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:33.049 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.050 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.050 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:33.050 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:55:33.050 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:55:33.050 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:55:33.050 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:55:33.050 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:55:33.050 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:55:33.051 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:55:33.051 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:55:33.051 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:55:33.051 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:55:33.051 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:55:33.051 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.051 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.051 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.051 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:55:33.051 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:55:33.051 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:55:33.052 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:55:33.052 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:55:33.052 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.052 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.052 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:55:33.052 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.052 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:55:33.052 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:55:33.052 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:55:33.052 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:55:33.052 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:55:33.052 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:55:33.053 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:55:33.053 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:55:33.053 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:55:33.053 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:55:33.053 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:55:33.053 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:55:33.053 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:55:33.053 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:55:33.053 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:55:33.053 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:55:33.053 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:55:33.053 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:55:33.053 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:55:33.053 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:55:33.054 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:55:33.054 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:55:33.054 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:55:33.054 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.054 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.054 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.054 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:55:33.058 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:55:33.058 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:55:33.058 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:55:33.058 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:55:33.058 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:55:33.058 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:55:33.058 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:55:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:55:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:55:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:55:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:55:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:55:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:55:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:55:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:55:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:55:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:55:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:55:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:55:33.059 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:55:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:55:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:55:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:55:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:55:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:55:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:55:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:55:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:55:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:55:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:55:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:55:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:55:33.060 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:55:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:55:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:55:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:55:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:55:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:55:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:55:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:55:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:55:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:55:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:55:33.061 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:55:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:55:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:55:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:55:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:55:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:55:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:55:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:55:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:55:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:55:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:55:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:55:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:55:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:55:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:55:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:55:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:55:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:55:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:55:33.063 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:55:33.064 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:55:33.064 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:55:33.064 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-04-25T11:55:33.077 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:55:33.286 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:55:33.286 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T11:55:33.287 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T11:55:33.287 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T11:55:33.287 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T11:55:33.287 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T11:55:33.287 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T11:55:33.287 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T11:55:33.287 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:55:33.287 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:55:33.287 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:55:33.287 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T11:55:33.287 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T11:55:33.287 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:55:33.287 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T11:55:33.288 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T11:55:33.288 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T11:55:33.288 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T11:55:33.288 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T11:55:33.288 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T11:55:33.288 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T11:55:33.288 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T11:55:33.288 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T11:55:33.288 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T11:55:33.288 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T11:55:33.288 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T11:55:33.288 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T11:55:33.288 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T11:55:33.289 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T11:55:33.289 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T11:55:33.300 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T11:55:33.300 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T11:55:33.300 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T11:55:33.300 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T11:55:33.300 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T11:55:33.300 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T11:55:33.301 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T11:55:33.301 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T11:55:33.301 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T11:55:33.301 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T11:55:33.301 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:55:33.301 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:55:33.301 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:55:33.301 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:55:33.301 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:55:33.301 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:55:33.301 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:55:33.301 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:55:33.301 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:55:33.302 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:55:33.302 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:55:33.302 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:55:33.302 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:55:33.302 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:55:33.302 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:55:33.302 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:55:33.302 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:55:33.302 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:55:33.302 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:55:33.302 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:55:33.302 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:55:33.302 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:55:33.303 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:55:33.303 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:55:33.303 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:55:33.303 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:55:33.303 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:55:33.303 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:55:33.303 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:55:33.303 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:55:33.303 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:55:33.303 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:55:33.303 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:55:33.303 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:55:33.303 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:55:33.304 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:55:33.304 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:55:33.304 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:55:33.304 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:55:33.304 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:55:33.304 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:55:33.304 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:55:33.304 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:55:33.304 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:55:33.304 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:55:33.304 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:55:33.304 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:55:33.304 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:55:33.305 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:55:33.305 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:55:33.305 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:55:33.305 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:55:33.305 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:55:33.305 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:55:33.305 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:55:33.305 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:55:33.305 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:55:33.306 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:55:33.306 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:55:33.306 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:55:33.306 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:55:33.306 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:55:33.306 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:55:33.306 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:55:33.306 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:55:33.306 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:55:33.306 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:55:33.306 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:55:33.306 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:55:33.306 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:55:33.307 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:55:33.307 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:55:33.307 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:55:33.307 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:55:33.307 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:55:33.307 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:55:33.307 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:55:33.307 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:55:33.307 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:55:33.307 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:55:33.307 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:55:33.307 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:55:33.307 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:55:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:55:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:55:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:55:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:55:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:55:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:55:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:55:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:55:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:55:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:55:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:55:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:55:33.308 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:55:33.309 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:55:33.309 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:55:33.309 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:55:33.309 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:55:33.309 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:55:33.309 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:55:33.309 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:55:33.309 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:55:33.309 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:55:33.309 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:55:33.309 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:55:33.309 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:55:33.309 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:55:33.310 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T11:55:33.310 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:55:33.310 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T11:55:33.310 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:55:33.310 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:55:33.310 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:55:33.310 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:55:33.435 DEBUG:teuthology.parallel:result is None 2024-04-25T11:55:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:55:35.528 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:55:35.528 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-04-25T11:55:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:55:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:55:35.529 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-25T11:55:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:55:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:55:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:55:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:55:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.529 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:35.530 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.531 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:55:35.532 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:55:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:55:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:55:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:55:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:55:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:55:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:55:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:55:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:55:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:55:35.533 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:55:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:55:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:55:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:55:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:55:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:55:35.534 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:55:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:55:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:55:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:55:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:55:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:55:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:55:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:55:35.535 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:55:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:55:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:55:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:55:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:55:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:55:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:55:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:55:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:55:35.536 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:55:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:55:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:55:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:55:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:55:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:55:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.537 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:55:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:55:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:55:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:55:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:55:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:55:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:55:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:55:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:55:35.538 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:55:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:55:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:55:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:55:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:55:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:55:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:55:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:55:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:55:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:55:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:55:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:55:35.539 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:55:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:55:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:55:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:55:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:55:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:55:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:55:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:55:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:55:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:55:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:55:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:55:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:55:35.540 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:55:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:55:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:55:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:55:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:55:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:55:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:55:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:55:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:55:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:55:35.541 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:55:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:55:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:55:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:55:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:55:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:55:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:55:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:55:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:55:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:55:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:55:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:55:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:55:35.542 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:55:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:55:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:55:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:55:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:55:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:55:35.543 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:55:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:55:35.544 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-25T11:55:35.544 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-25T11:55:35.925 DEBUG:teuthology.parallel:result is None 2024-04-25T11:55:38.754 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:55:38.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T11:55:38.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T11:55:38.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T11:55:38.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T11:55:38.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T11:55:38.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T11:55:38.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T11:55:38.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:55:38.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:55:38.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:55:38.755 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T11:55:38.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T11:55:38.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:55:38.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T11:55:38.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T11:55:38.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T11:55:38.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T11:55:38.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T11:55:38.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T11:55:38.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T11:55:38.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T11:55:38.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T11:55:38.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T11:55:38.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T11:55:38.756 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T11:55:38.757 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T11:55:38.757 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T11:55:38.757 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T11:55:38.757 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T11:55:38.757 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T11:55:38.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T11:55:38.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T11:55:38.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T11:55:38.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T11:55:38.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T11:55:38.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T11:55:38.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T11:55:38.768 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T11:55:38.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T11:55:38.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:55:38.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:55:38.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:55:38.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:55:38.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:55:38.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:55:38.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:55:38.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:55:38.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:55:38.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:55:38.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:55:38.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:55:38.769 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:55:38.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:55:38.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:55:38.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:55:38.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:55:38.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:55:38.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:55:38.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:55:38.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:55:38.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:55:38.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:55:38.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:55:38.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:55:38.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:55:38.770 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:55:38.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:55:38.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:55:38.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:55:38.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:55:38.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:55:38.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:55:38.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:55:38.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:55:38.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:55:38.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:55:38.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:55:38.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:55:38.771 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:55:38.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:55:38.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:55:38.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:55:38.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:55:38.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:55:38.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:55:38.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:55:38.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:55:38.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:55:38.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:55:38.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:55:38.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:55:38.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:55:38.772 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:55:38.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:55:38.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:55:38.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:55:38.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:55:38.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:55:38.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:55:38.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:55:38.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:55:38.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:55:38.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:55:38.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:55:38.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:55:38.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:55:38.773 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:55:38.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:55:38.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:55:38.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:55:38.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:55:38.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:55:38.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:55:38.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:55:38.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:55:38.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:55:38.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:55:38.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:55:38.774 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:55:38.775 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:55:38.775 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:55:38.775 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:55:38.775 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:55:38.775 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:55:38.775 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:55:38.775 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:55:38.775 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:55:38.775 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:55:38.775 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:55:38.775 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:55:38.775 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:55:38.775 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:55:38.776 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:55:38.776 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:55:38.776 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:55:38.776 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:55:38.776 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:55:38.776 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:55:38.776 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:55:38.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:55:38.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:55:38.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:55:38.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:55:38.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:55:38.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:55:38.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:55:38.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:55:38.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:55:38.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T11:55:38.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:55:38.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T11:55:38.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:55:38.777 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:55:38.778 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:55:38.778 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:55:40.954 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:55:40.954 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:55:40.954 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-04-25T11:55:40.954 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.955 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.955 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:55:40.955 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:55:40.955 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-04-25T11:55:40.955 INFO:teuthology.orchestra.run.smithi169.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:55:40.955 INFO:teuthology.orchestra.run.smithi169.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:55:40.955 INFO:teuthology.orchestra.run.smithi169.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:55:40.955 INFO:teuthology.orchestra.run.smithi169.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:55:40.955 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.955 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.955 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.955 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.955 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:40.956 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.956 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.956 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.956 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:40.956 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:40.956 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:40.956 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:40.956 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:40.956 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.956 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.956 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:40.956 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.957 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.957 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.957 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:40.957 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:40.957 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.957 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.957 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:55:40.957 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:55:40.957 INFO:teuthology.orchestra.run.smithi169.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:55:40.957 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:55:40.957 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:55:40.957 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:55:40.957 INFO:teuthology.orchestra.run.smithi169.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:55:40.957 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:55:40.958 INFO:teuthology.orchestra.run.smithi169.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:55:40.958 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:55:40.958 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:55:40.958 INFO:teuthology.orchestra.run.smithi169.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:55:40.958 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.958 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.958 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.958 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:55:40.958 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:55:40.958 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:55:40.958 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:55:40.958 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:55:40.959 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.959 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.959 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:55:40.959 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.959 INFO:teuthology.orchestra.run.smithi169.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:55:40.959 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:55:40.959 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:55:40.959 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:55:40.959 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:55:40.959 INFO:teuthology.orchestra.run.smithi169.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:55:40.959 INFO:teuthology.orchestra.run.smithi169.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:55:40.959 INFO:teuthology.orchestra.run.smithi169.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:55:40.959 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:55:40.959 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:55:40.960 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:55:40.960 INFO:teuthology.orchestra.run.smithi169.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:55:40.960 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:55:40.960 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:55:40.960 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:55:40.960 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:55:40.960 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:55:40.960 INFO:teuthology.orchestra.run.smithi169.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:55:40.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:55:40.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:55:40.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:55:40.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:55:40.960 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:55:40.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:55:40.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:55:40.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:55:40.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:55:40.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:55:40.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:55:40.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:55:40.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:55:40.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:55:40.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:55:40.961 INFO:teuthology.orchestra.run.smithi169.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:55:40.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:55:40.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:55:40.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:55:40.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:55:40.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:55:40.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:55:40.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:55:40.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:55:40.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:55:40.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:55:40.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:55:40.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:55:40.962 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:55:40.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:55:40.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:55:40.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:55:40.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:55:40.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:55:40.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:55:40.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:55:40.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:55:40.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:55:40.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:55:40.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:55:40.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:55:40.963 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:55:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:55:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:55:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:55:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:55:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:55:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:55:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:55:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:55:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:55:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:55:40.964 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:55:40.965 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:55:40.965 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:55:40.965 INFO:teuthology.orchestra.run.smithi169.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:55:40.965 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:55:40.965 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:55:40.965 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:55:40.965 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:55:40.965 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:55:40.965 INFO:teuthology.orchestra.run.smithi169.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:55:40.965 INFO:teuthology.orchestra.run.smithi169.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:55:40.965 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.965 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.965 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:55:40.966 INFO:teuthology.orchestra.run.smithi169.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:55:40.966 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:55:40.966 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:55:40.966 INFO:teuthology.orchestra.run.smithi169.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:55:40.966 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:55:40.966 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:55:40.966 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:55:40.966 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:55:40.966 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-04-25T11:55:41.367 DEBUG:teuthology.parallel:result is None 2024-04-25T11:55:41.368 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:55:41.662 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:55:41.689 INFO:teuthology.orchestra.run.smithi019.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T11:55:41.690 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T11:55:41.690 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T11:55:41.691 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:55:41.827 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:55:41.853 INFO:teuthology.orchestra.run.smithi150.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T11:55:41.853 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T11:55:41.853 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T11:55:41.855 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:55:41.991 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:55:42.012 INFO:teuthology.orchestra.run.smithi169.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T11:55:42.012 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T11:55:42.012 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T11:55:42.014 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T11:55:42.015 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:55:42.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:55:42.050 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T11:55:42.050 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:55:42.082 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:55:42.082 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:55:42.111 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T11:55:42.111 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:55:42.111 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:55:42.144 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:55:42.214 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T11:55:42.214 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:55:42.243 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:55:42.316 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:55:42.316 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:55:42.344 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:55:42.411 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T11:55:42.411 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:55:42.411 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:55:42.442 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:55:42.516 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T11:55:42.516 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:55:42.545 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:55:42.614 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:55:42.614 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:55:42.639 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:55:42.704 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T11:55:42.704 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:55:42.704 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:55:42.733 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:55:42.803 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T11:55:42.803 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:55:42.832 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:55:42.900 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:55:42.900 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:55:42.923 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:55:42.986 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T11:55:43.099 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client dirsize rbytes': False, 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2}} 2024-04-25T11:55:43.100 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:55:43.100 INFO:tasks.cephadm:Cluster fsid is be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:55:43.100 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T11:55:43.100 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.19', 'mon.b': '172.21.15.150', 'mon.c': '172.21.15.169'} 2024-04-25T11:55:43.101 INFO:tasks.cephadm:First mon is mon.a on smithi019 2024-04-25T11:55:43.101 INFO:tasks.cephadm:First mgr is x 2024-04-25T11:55:43.101 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T11:55:43.101 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname $(hostname -s) 2024-04-25T11:55:43.131 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2024-04-25T11:55:43.160 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname $(hostname -s) 2024-04-25T11:55:43.184 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T11:55:43.184 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:55:43.327 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T11:55:43.543 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:55:43.545 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T11:55:43.545 DEBUG:teuthology.orchestra.run.smithi019:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T11:55:43.963 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 11:55 /home/ubuntu/cephtest/cephadm 2024-04-25T11:55:43.963 DEBUG:teuthology.orchestra.run.smithi150:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T11:55:44.556 INFO:teuthology.orchestra.run.smithi150.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 11:55 /home/ubuntu/cephtest/cephadm 2024-04-25T11:55:44.556 DEBUG:teuthology.orchestra.run.smithi169:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T11:55:44.974 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 11:55 /home/ubuntu/cephtest/cephadm 2024-04-25T11:55:44.974 DEBUG:teuthology.orchestra.run.smithi019:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T11:55:44.994 DEBUG:teuthology.orchestra.run.smithi150:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T11:55:45.013 DEBUG:teuthology.orchestra.run.smithi169:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T11:55:45.048 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T11:55:45.049 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T11:55:45.051 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T11:55:45.055 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T11:55:46.678 INFO:teuthology.orchestra.run.smithi019.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T11:55:46.818 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T11:55:46.838 INFO:teuthology.orchestra.run.smithi169.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T11:56:15.320 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T11:56:15.320 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T11:56:15.320 INFO:teuthology.orchestra.run.smithi019.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T11:56:15.320 INFO:teuthology.orchestra.run.smithi019.stdout: "repo_digests": [ 2024-04-25T11:56:15.320 INFO:teuthology.orchestra.run.smithi019.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T11:56:15.320 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T11:56:15.320 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T11:56:29.934 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-25T11:56:29.934 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T11:56:29.934 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T11:56:29.934 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2024-04-25T11:56:29.935 INFO:teuthology.orchestra.run.smithi150.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T11:56:29.935 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-04-25T11:56:29.935 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-25T11:56:42.288 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T11:56:42.288 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T11:56:42.288 INFO:teuthology.orchestra.run.smithi169.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T11:56:42.288 INFO:teuthology.orchestra.run.smithi169.stdout: "repo_digests": [ 2024-04-25T11:56:42.289 INFO:teuthology.orchestra.run.smithi169.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T11:56:42.289 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2024-04-25T11:56:42.289 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T11:56:42.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph 2024-04-25T11:56:42.347 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2024-04-25T11:56:42.380 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph 2024-04-25T11:56:42.407 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /etc/ceph 2024-04-25T11:56:42.437 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2024-04-25T11:56:42.464 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /etc/ceph 2024-04-25T11:56:42.488 INFO:tasks.cephadm:Writing seed config... 2024-04-25T11:56:42.489 INFO:tasks.cephadm: override: [client] client dirsize rbytes = False 2024-04-25T11:56:42.490 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T11:56:42.490 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T11:56:42.490 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T11:56:42.490 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T11:56:42.490 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T11:56:42.490 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T11:56:42.490 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T11:56:42.490 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T11:56:42.490 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T11:56:42.490 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T11:56:42.491 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T11:56:42.491 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T11:56:42.491 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T11:56:42.491 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T11:56:42.491 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T11:56:42.491 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T11:56:42.491 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T11:56:42.491 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T11:56:42.491 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T11:56:42.491 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T11:56:42.491 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T11:56:42.491 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T11:56:42.491 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T11:56:42.491 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T11:56:42.492 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T11:56:42.492 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T11:56:42.492 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T11:56:42.492 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T11:56:42.492 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T11:56:42.492 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T11:56:42.492 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T11:56:42.492 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T11:56:42.492 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-25T11:56:42.492 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T11:56:42.492 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-25T11:56:42.492 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-25T11:56:42.492 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T11:56:42.493 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T11:56:42.493 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-25T11:56:42.493 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T11:56:42.493 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T11:56:42.493 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T11:56:42.493 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T11:56:42.493 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T11:56:42.493 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T11:56:42.493 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T11:56:42.493 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T11:56:42.494 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:56:42.494 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T11:56:42.513 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = be62e290-02fa-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client dirsize rbytes = False client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T11:56:42.513 DEBUG:teuthology.orchestra.run.smithi019:mon.a> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T11:56:42.556 DEBUG:teuthology.orchestra.run.smithi019:mgr.x> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T11:56:42.599 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T11:56:42.599 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid be62e290-02fa-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.19 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:56:42.803 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:56:42.803 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', 'be62e290-02fa-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.19', '--skip-admin-label'] 2024-04-25T11:56:42.803 INFO:teuthology.orchestra.run.smithi019.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T11:56:42.804 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying podman|docker is present... 2024-04-25T11:56:43.014 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T11:56:43.014 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying lvm2 is present... 2024-04-25T11:56:43.014 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying time synchronization is in place... 2024-04-25T11:56:43.023 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T11:56:43.024 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T11:56:43.031 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T11:56:43.031 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T11:56:43.039 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-04-25T11:56:43.047 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-04-25T11:56:43.047 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-04-25T11:56:43.047 INFO:teuthology.orchestra.run.smithi019.stdout:Repeating the final host check... 2024-04-25T11:56:43.072 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T11:56:43.072 INFO:teuthology.orchestra.run.smithi019.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T11:56:43.072 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl is present 2024-04-25T11:56:43.072 INFO:teuthology.orchestra.run.smithi019.stdout:lvcreate is present 2024-04-25T11:56:43.088 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T11:56:43.088 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T11:56:43.094 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T11:56:43.095 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T11:56:43.102 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-04-25T11:56:43.109 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-04-25T11:56:43.109 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-04-25T11:56:43.109 INFO:teuthology.orchestra.run.smithi019.stdout:Host looks OK 2024-04-25T11:56:43.109 INFO:teuthology.orchestra.run.smithi019.stdout:Cluster fsid: be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:56:43.109 INFO:teuthology.orchestra.run.smithi019.stdout:Acquiring lock 139636607511712 on /run/cephadm/be62e290-02fa-11ef-bc93-c7b262605968.lock 2024-04-25T11:56:43.109 INFO:teuthology.orchestra.run.smithi019.stdout:Lock 139636607511712 acquired on /run/cephadm/be62e290-02fa-11ef-bc93-c7b262605968.lock 2024-04-25T11:56:43.109 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 3300 ... 2024-04-25T11:56:43.110 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 6789 ... 2024-04-25T11:56:43.110 INFO:teuthology.orchestra.run.smithi019.stdout:Base mon IP(s) is [172.21.15.19:3300, 172.21.15.19:6789], mon addrv is [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-04-25T11:56:43.113 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.19 metric 100 2024-04-25T11:56:43.113 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.19 metric 100 2024-04-25T11:56:43.116 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T11:56:43.116 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T11:56:43.116 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T11:56:43.120 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T11:56:43.120 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T11:56:43.120 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T11:56:43.120 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T11:56:43.120 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T11:56:43.120 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T11:56:43.120 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-04-25T11:56:43.120 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-04-25T11:56:43.121 INFO:teuthology.orchestra.run.smithi019.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T11:56:43.121 INFO:teuthology.orchestra.run.smithi019.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T11:56:43.121 INFO:teuthology.orchestra.run.smithi019.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T11:56:46.064 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T11:56:46.064 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T11:56:46.064 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T11:56:46.064 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T11:56:46.064 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T11:56:46.065 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T11:56:46.065 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T11:56:52.039 INFO:teuthology.orchestra.run.smithi019.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T11:56:52.040 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T11:56:52.040 INFO:teuthology.orchestra.run.smithi019.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T11:56:53.037 INFO:teuthology.orchestra.run.smithi019.stdout:stat: stdout 167 167 2024-04-25T11:56:53.038 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial keys... 2024-04-25T11:56:54.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQAFRSpmgiTYFhAAxWnD3OZGVzA+sTxfFnqU+Q== 2024-04-25T11:56:55.163 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQAGRSpmycI6JRAAZypgYUwB9WQ+g4UacaYatQ== 2024-04-25T11:56:56.194 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQAHRSpmglSxGxAA9nu5jI15cIqJtcjJjaXIUw== 2024-04-25T11:56:56.195 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial monmap... 2024-04-25T11:56:57.416 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T11:56:57.417 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T11:56:57.417 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:56:57.417 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T11:56:57.417 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool for a [v2:172.21.15.19:3300,v1:172.21.15.19:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T11:56:57.417 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = quincy 2024-04-25T11:56:57.417 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: set fsid to be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:56:57.417 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T11:56:57.417 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:56:57.417 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mon... 2024-04-25T11:56:58.564 INFO:teuthology.orchestra.run.smithi019.stdout:create mon.a on 2024-04-25T11:56:58.765 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T11:56:58.916 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T11:56:59.089 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-be62e290-02fa-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-be62e290-02fa-11ef-bc93-c7b262605968.target. 2024-04-25T11:56:59.089 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-be62e290-02fa-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-be62e290-02fa-11ef-bc93-c7b262605968.target. 2024-04-25T11:56:59.364 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-be62e290-02fa-11ef-bc93-c7b262605968@mon.a 2024-04-25T11:56:59.364 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-be62e290-02fa-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-be62e290-02fa-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T11:56:59.546 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-be62e290-02fa-11ef-bc93-c7b262605968.target.wants/ceph-be62e290-02fa-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-be62e290-02fa-11ef-bc93-c7b262605968@.service. 2024-04-25T11:57:00.113 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T11:57:00.113 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-25T11:57:00.120 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T11:57:00.120 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T11:57:00.120 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-25T11:57:00.121 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T11:57:00.121 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon to start... 2024-04-25T11:57:00.121 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon... 2024-04-25T11:57:01.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T11:57:01.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout id: be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:57:01.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T11:57:01.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T11:57:01.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout services: 2024-04-25T11:57:01.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.572591s) 2024-04-25T11:57:01.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T11:57:01.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T11:57:01.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T11:57:01.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout data: 2024-04-25T11:57:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T11:57:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T11:57:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T11:57:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T11:57:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T11:57:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.573+0000 7f5a71304700 1 Processor -- start 2024-04-25T11:57:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.573+0000 7f5a71304700 1 -- start start 2024-04-25T11:57:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.574+0000 7f5a71304700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c108650 0x7f5a6c108a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.574+0000 7f5a71304700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c109040 con 0x7f5a6c108650 2024-04-25T11:57:01.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.574+0000 7f5a6affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c108650 0x7f5a6c108a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.574+0000 7f5a6affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c108650 0x7f5a6c108a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52264/0 (socket says 172.21.15.19:52264) 2024-04-25T11:57:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.574+0000 7f5a6affd700 1 -- 172.21.15.19:0/3270058416 learned_addr learned my addr 172.21.15.19:0/3270058416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.574+0000 7f5a6affd700 1 -- 172.21.15.19:0/3270058416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c109800 con 0x7f5a6c108650 2024-04-25T11:57:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.574+0000 7f5a6affd700 1 --2- 172.21.15.19:0/3270058416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c108650 0x7f5a6c108a70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0098a0 tx=0x7f5a5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4160be49e757e767 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.575+0000 7f5a69ffb700 1 -- 172.21.15.19:0/3270058416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a5c004160 con 0x7f5a6c108650 2024-04-25T11:57:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.575+0000 7f5a69ffb700 1 -- 172.21.15.19:0/3270058416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5c004300 con 0x7f5a6c108650 2024-04-25T11:57:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.575+0000 7f5a69ffb700 1 -- 172.21.15.19:0/3270058416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a5c0036a0 con 0x7f5a6c108650 2024-04-25T11:57:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.575+0000 7f5a71304700 1 -- 172.21.15.19:0/3270058416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c108650 msgr2=0x7f5a6c108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.575+0000 7f5a71304700 1 --2- 172.21.15.19:0/3270058416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c108650 0x7f5a6c108a70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0098a0 tx=0x7f5a5c009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.575+0000 7f5a71304700 1 -- 172.21.15.19:0/3270058416 shutdown_connections 2024-04-25T11:57:01.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.575+0000 7f5a71304700 1 --2- 172.21.15.19:0/3270058416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c108650 0x7f5a6c108a70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.575+0000 7f5a71304700 1 -- 172.21.15.19:0/3270058416 >> 172.21.15.19:0/3270058416 conn(0x7f5a6c103cb0 msgr2=0x7f5a6c1060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.575+0000 7f5a71304700 1 -- 172.21.15.19:0/3270058416 shutdown_connections 2024-04-25T11:57:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.575+0000 7f5a71304700 1 -- 172.21.15.19:0/3270058416 wait complete. 2024-04-25T11:57:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.576+0000 7f5a71304700 1 Processor -- start 2024-04-25T11:57:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.576+0000 7f5a71304700 1 -- start start 2024-04-25T11:57:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.576+0000 7f5a71304700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c108650 0x7f5a6c07db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.576+0000 7f5a71304700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c109d30 con 0x7f5a6c108650 2024-04-25T11:57:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.576+0000 7f5a6affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c108650 0x7f5a6c07db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.576+0000 7f5a6affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c108650 0x7f5a6c07db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52280/0 (socket says 172.21.15.19:52280) 2024-04-25T11:57:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.576+0000 7f5a6affd700 1 -- 172.21.15.19:0/1040730859 learned_addr learned my addr 172.21.15.19:0/1040730859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.577+0000 7f5a6affd700 1 -- 172.21.15.19:0/1040730859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c07e0a0 con 0x7f5a6c108650 2024-04-25T11:57:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.577+0000 7f5a6affd700 1 --2- 172.21.15.19:0/1040730859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c108650 0x7f5a6c07db60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c000c00 tx=0x7f5a5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:01.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.577+0000 7f5a63fff700 1 -- 172.21.15.19:0/1040730859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a5c003db0 con 0x7f5a6c108650 2024-04-25T11:57:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.577+0000 7f5a63fff700 1 -- 172.21.15.19:0/1040730859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5c011630 con 0x7f5a6c108650 2024-04-25T11:57:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.577+0000 7f5a71304700 1 -- 172.21.15.19:0/1040730859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6c07c290 con 0x7f5a6c108650 2024-04-25T11:57:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.577+0000 7f5a63fff700 1 -- 172.21.15.19:0/1040730859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a5c011950 con 0x7f5a6c108650 2024-04-25T11:57:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.577+0000 7f5a71304700 1 -- 172.21.15.19:0/1040730859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c07c7d0 con 0x7f5a6c108650 2024-04-25T11:57:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.577+0000 7f5a63fff700 1 -- 172.21.15.19:0/1040730859 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5a5c01a460 con 0x7f5a6c108650 2024-04-25T11:57:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.577+0000 7f5a63fff700 1 -- 172.21.15.19:0/1040730859 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5a5c01b780 con 0x7f5a6c108650 2024-04-25T11:57:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.578+0000 7f5a71304700 1 -- 172.21.15.19:0/1040730859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a6c0640a0 con 0x7f5a6c108650 2024-04-25T11:57:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.580+0000 7f5a63fff700 1 -- 172.21.15.19:0/1040730859 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5a5c031020 con 0x7f5a6c108650 2024-04-25T11:57:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.653+0000 7f5a71304700 1 -- 172.21.15.19:0/1040730859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f5a6c195110 con 0x7f5a6c108650 2024-04-25T11:57:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.653+0000 7f5a63fff700 1 -- 172.21.15.19:0/1040730859 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f5a5c01ba20 con 0x7f5a6c108650 2024-04-25T11:57:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.654+0000 7f5a71304700 1 -- 172.21.15.19:0/1040730859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c108650 msgr2=0x7f5a6c07db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:01.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.654+0000 7f5a71304700 1 --2- 172.21.15.19:0/1040730859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c108650 0x7f5a6c07db60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c000c00 tx=0x7f5a5c004750 comp rx=0 tx=0).stop 2024-04-25T11:57:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.654+0000 7f5a71304700 1 -- 172.21.15.19:0/1040730859 shutdown_connections 2024-04-25T11:57:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.654+0000 7f5a71304700 1 --2- 172.21.15.19:0/1040730859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a6c108650 0x7f5a6c07db60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.654+0000 7f5a71304700 1 -- 172.21.15.19:0/1040730859 >> 172.21.15.19:0/1040730859 conn(0x7f5a6c103cb0 msgr2=0x7f5a6c192870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.654+0000 7f5a71304700 1 -- 172.21.15.19:0/1040730859 shutdown_connections 2024-04-25T11:57:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:00.654+0000 7f5a71304700 1 -- 172.21.15.19:0/1040730859 wait complete. 2024-04-25T11:57:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:mon is available 2024-04-25T11:57:01.177 INFO:teuthology.orchestra.run.smithi019.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T11:57:02.267 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:02 smithi019 ceph-mon[28843]: from='client.? 172.21.15.19:0/2555227855' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T11:57:02.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T11:57:02.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T11:57:02.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:57:02.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T11:57:02.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-04-25T11:57:02.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T11:57:02.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T11:57:02.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T11:57:02.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T11:57:02.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T11:57:02.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T11:57:02.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T11:57:02.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T11:57:02.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T11:57:02.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T11:57:02.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T11:57:02.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T11:57:02.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T11:57:02.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T11:57:02.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.587+0000 7ff1262d5700 1 Processor -- start 2024-04-25T11:57:02.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.587+0000 7ff1262d5700 1 -- start start 2024-04-25T11:57:02.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.587+0000 7ff1262d5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff12007de40 0x7ff12007c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:02.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.587+0000 7ff1262d5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff12007c8c0 con 0x7ff12007de40 2024-04-25T11:57:02.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.587+0000 7ff11ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff12007de40 0x7ff12007c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:02.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.588+0000 7ff11ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff12007de40 0x7ff12007c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52296/0 (socket says 172.21.15.19:52296) 2024-04-25T11:57:02.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.588+0000 7ff11ffff700 1 -- 172.21.15.19:0/3372163289 learned_addr learned my addr 172.21.15.19:0/3372163289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:02.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.588+0000 7ff11ffff700 1 -- 172.21.15.19:0/3372163289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12007ca40 con 0x7ff12007de40 2024-04-25T11:57:02.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.588+0000 7ff11ffff700 1 --2- 172.21.15.19:0/3372163289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff12007de40 0x7ff12007c2f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff1080098a0 tx=0x7ff108009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dddced67af5c4c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:02.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.588+0000 7ff11effd700 1 -- 172.21.15.19:0/3372163289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff108004160 con 0x7ff12007de40 2024-04-25T11:57:02.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.588+0000 7ff11effd700 1 -- 172.21.15.19:0/3372163289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff108004300 con 0x7ff12007de40 2024-04-25T11:57:02.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.588+0000 7ff11effd700 1 -- 172.21.15.19:0/3372163289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1080036a0 con 0x7ff12007de40 2024-04-25T11:57:02.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.589+0000 7ff1262d5700 1 -- 172.21.15.19:0/3372163289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff12007de40 msgr2=0x7ff12007c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:02.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.589+0000 7ff1262d5700 1 --2- 172.21.15.19:0/3372163289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff12007de40 0x7ff12007c2f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff1080098a0 tx=0x7ff108009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:02.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.589+0000 7ff1262d5700 1 -- 172.21.15.19:0/3372163289 shutdown_connections 2024-04-25T11:57:02.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.589+0000 7ff1262d5700 1 --2- 172.21.15.19:0/3372163289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff12007de40 0x7ff12007c2f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:02.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.589+0000 7ff1262d5700 1 -- 172.21.15.19:0/3372163289 >> 172.21.15.19:0/3372163289 conn(0x7ff120103bb0 msgr2=0x7ff120105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:02.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.589+0000 7ff1262d5700 1 -- 172.21.15.19:0/3372163289 shutdown_connections 2024-04-25T11:57:02.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.589+0000 7ff1262d5700 1 -- 172.21.15.19:0/3372163289 wait complete. 2024-04-25T11:57:02.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.589+0000 7ff1262d5700 1 Processor -- start 2024-04-25T11:57:02.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.589+0000 7ff1262d5700 1 -- start start 2024-04-25T11:57:02.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.590+0000 7ff1262d5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff12007de40 0x7ff12019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:02.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.590+0000 7ff1262d5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff12007cf70 con 0x7ff12007de40 2024-04-25T11:57:02.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.590+0000 7ff11ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff12007de40 0x7ff12019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:02.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.590+0000 7ff11ffff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff12007de40 0x7ff12019dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52302/0 (socket says 172.21.15.19:52302) 2024-04-25T11:57:02.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.590+0000 7ff11ffff700 1 -- 172.21.15.19:0/2555227855 learned_addr learned my addr 172.21.15.19:0/2555227855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:02.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.590+0000 7ff11ffff700 1 -- 172.21.15.19:0/2555227855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12019e290 con 0x7ff12007de40 2024-04-25T11:57:02.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.590+0000 7ff11ffff700 1 --2- 172.21.15.19:0/2555227855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff12007de40 0x7ff12019dd50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff108000c00 tx=0x7ff108004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:02.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.590+0000 7ff11d7fa700 1 -- 172.21.15.19:0/2555227855 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff108003db0 con 0x7ff12007de40 2024-04-25T11:57:02.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.590+0000 7ff11d7fa700 1 -- 172.21.15.19:0/2555227855 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff108011630 con 0x7ff12007de40 2024-04-25T11:57:02.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.591+0000 7ff1262d5700 1 -- 172.21.15.19:0/2555227855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12019e520 con 0x7ff12007de40 2024-04-25T11:57:02.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.591+0000 7ff11d7fa700 1 -- 172.21.15.19:0/2555227855 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff108011950 con 0x7ff12007de40 2024-04-25T11:57:02.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.591+0000 7ff1262d5700 1 -- 172.21.15.19:0/2555227855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1201a1210 con 0x7ff12007de40 2024-04-25T11:57:02.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.591+0000 7ff11d7fa700 1 -- 172.21.15.19:0/2555227855 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff108011af0 con 0x7ff12007de40 2024-04-25T11:57:02.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.591+0000 7ff11d7fa700 1 -- 172.21.15.19:0/2555227855 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff10801c450 con 0x7ff12007de40 2024-04-25T11:57:02.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.592+0000 7ff1262d5700 1 -- 172.21.15.19:0/2555227855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1200640a0 con 0x7ff12007de40 2024-04-25T11:57:02.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.594+0000 7ff11d7fa700 1 -- 172.21.15.19:0/2555227855 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff108032020 con 0x7ff12007de40 2024-04-25T11:57:02.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.664+0000 7ff1262d5700 1 -- 172.21.15.19:0/2555227855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff1201a13e0 con 0x7ff12007de40 2024-04-25T11:57:02.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.667+0000 7ff11d7fa700 1 -- 172.21.15.19:0/2555227855 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7ff108004260 con 0x7ff12007de40 2024-04-25T11:57:02.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.668+0000 7ff11d7fa700 1 -- 172.21.15.19:0/2555227855 <== mon.0 v2:172.21.15.19:3300/0 8 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7ff10801c650 con 0x7ff12007de40 2024-04-25T11:57:02.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.669+0000 7ff1262d5700 1 -- 172.21.15.19:0/2555227855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff12007de40 msgr2=0x7ff12019dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:02.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.669+0000 7ff1262d5700 1 --2- 172.21.15.19:0/2555227855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff12007de40 0x7ff12019dd50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff108000c00 tx=0x7ff108004750 comp rx=0 tx=0).stop 2024-04-25T11:57:02.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.669+0000 7ff1262d5700 1 -- 172.21.15.19:0/2555227855 shutdown_connections 2024-04-25T11:57:02.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.669+0000 7ff1262d5700 1 --2- 172.21.15.19:0/2555227855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff12007de40 0x7ff12019dd50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:02.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.669+0000 7ff1262d5700 1 -- 172.21.15.19:0/2555227855 >> 172.21.15.19:0/2555227855 conn(0x7ff120103bb0 msgr2=0x7ff120104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:02.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.669+0000 7ff1262d5700 1 -- 172.21.15.19:0/2555227855 shutdown_connections 2024-04-25T11:57:02.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:01.669+0000 7ff1262d5700 1 -- 172.21.15.19:0/2555227855 wait complete. 2024-04-25T11:57:02.281 INFO:teuthology.orchestra.run.smithi019.stdout:Generating new minimal ceph.conf... 2024-04-25T11:57:03.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.727+0000 7f1c10cad700 1 Processor -- start 2024-04-25T11:57:03.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.727+0000 7f1c10cad700 1 -- start start 2024-04-25T11:57:03.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.727+0000 7f1c10cad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0c1074a0 0x7f1c0c1078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:03.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.727+0000 7f1c10cad700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c107e90 con 0x7f1c0c1074a0 2024-04-25T11:57:03.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.728+0000 7f1c0a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0c1074a0 0x7f1c0c1078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:03.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.728+0000 7f1c0a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0c1074a0 0x7f1c0c1078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44700/0 (socket says 172.21.15.19:44700) 2024-04-25T11:57:03.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.728+0000 7f1c0a59c700 1 -- 172.21.15.19:0/46523464 learned_addr learned my addr 172.21.15.19:0/46523464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:03.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.728+0000 7f1c0a59c700 1 -- 172.21.15.19:0/46523464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0c1086a0 con 0x7f1c0c1074a0 2024-04-25T11:57:03.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.728+0000 7f1c0a59c700 1 --2- 172.21.15.19:0/46523464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0c1074a0 0x7f1c0c1078c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0098a0 tx=0x7f1bfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4c76398499f6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:03.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.728+0000 7f1c0959a700 1 -- 172.21.15.19:0/46523464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bfc004160 con 0x7f1c0c1074a0 2024-04-25T11:57:03.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.728+0000 7f1c0959a700 1 -- 172.21.15.19:0/46523464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f1bfc004300 con 0x7f1c0c1074a0 2024-04-25T11:57:03.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.728+0000 7f1c0959a700 1 -- 172.21.15.19:0/46523464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bfc011b20 con 0x7f1c0c1074a0 2024-04-25T11:57:03.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.729+0000 7f1c10cad700 1 -- 172.21.15.19:0/46523464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0c1074a0 msgr2=0x7f1c0c1078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:03.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.729+0000 7f1c10cad700 1 --2- 172.21.15.19:0/46523464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0c1074a0 0x7f1c0c1078c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0098a0 tx=0x7f1bfc009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:03.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.729+0000 7f1c10cad700 1 -- 172.21.15.19:0/46523464 shutdown_connections 2024-04-25T11:57:03.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.729+0000 7f1c10cad700 1 --2- 172.21.15.19:0/46523464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0c1074a0 0x7f1c0c1078c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:03.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.729+0000 7f1c10cad700 1 -- 172.21.15.19:0/46523464 >> 172.21.15.19:0/46523464 conn(0x7f1c0c102a40 msgr2=0x7f1c0c104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:03.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.729+0000 7f1c10cad700 1 -- 172.21.15.19:0/46523464 shutdown_connections 2024-04-25T11:57:03.333 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.729+0000 7f1c10cad700 1 -- 172.21.15.19:0/46523464 wait complete. 2024-04-25T11:57:03.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.730+0000 7f1c10cad700 1 Processor -- start 2024-04-25T11:57:03.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.730+0000 7f1c10cad700 1 -- start start 2024-04-25T11:57:03.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.730+0000 7f1c10cad700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0c1074a0 0x7f1c0c19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:03.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.730+0000 7f1c10cad700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c10ba50 con 0x7f1c0c1074a0 2024-04-25T11:57:03.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.730+0000 7f1c0a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0c1074a0 0x7f1c0c19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:03.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.730+0000 7f1c0a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0c1074a0 0x7f1c0c19c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44710/0 (socket says 172.21.15.19:44710) 2024-04-25T11:57:03.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.730+0000 7f1c0a59c700 1 -- 172.21.15.19:0/3112339054 learned_addr learned my addr 172.21.15.19:0/3112339054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:03.334 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.731+0000 7f1c0a59c700 1 -- 172.21.15.19:0/3112339054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0c19c9a0 con 0x7f1c0c1074a0 2024-04-25T11:57:03.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.731+0000 7f1c0a59c700 1 --2- 172.21.15.19:0/3112339054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0c1074a0 0x7f1c0c19c460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0117e0 tx=0x7f1bfc011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:03.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.731+0000 7f1bfb7fe700 1 -- 172.21.15.19:0/3112339054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bfc003b30 con 0x7f1c0c1074a0 2024-04-25T11:57:03.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.731+0000 7f1bfb7fe700 1 -- 172.21.15.19:0/3112339054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f1bfc003cd0 con 0x7f1c0c1074a0 2024-04-25T11:57:03.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.731+0000 7f1c10cad700 1 -- 172.21.15.19:0/3112339054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0c19cc30 con 0x7f1c0c1074a0 2024-04-25T11:57:03.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.731+0000 7f1bfb7fe700 1 -- 172.21.15.19:0/3112339054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bfc01a830 con 0x7f1c0c1074a0 2024-04-25T11:57:03.335 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.731+0000 7f1c10cad700 1 -- 172.21.15.19:0/3112339054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0c19d090 con 0x7f1c0c1074a0 2024-04-25T11:57:03.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.731+0000 7f1bfb7fe700 1 -- 172.21.15.19:0/3112339054 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1bfc02d3f0 con 0x7f1c0c1074a0 2024-04-25T11:57:03.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.731+0000 7f1bfb7fe700 1 -- 172.21.15.19:0/3112339054 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1bfc019700 con 0x7f1c0c1074a0 2024-04-25T11:57:03.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.732+0000 7f1c10cad700 1 -- 172.21.15.19:0/3112339054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c0c0640a0 con 0x7f1c0c1074a0 2024-04-25T11:57:03.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.734+0000 7f1bfb7fe700 1 -- 172.21.15.19:0/3112339054 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1bfc024ba0 con 0x7f1c0c1074a0 2024-04-25T11:57:03.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.806+0000 7f1c10cad700 1 -- 172.21.15.19:0/3112339054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f1c0c02e330 con 0x7f1c0c1074a0 2024-04-25T11:57:03.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.806+0000 7f1bfb7fe700 1 -- 172.21.15.19:0/3112339054 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f1bfc032030 con 0x7f1c0c1074a0 2024-04-25T11:57:03.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.807+0000 7f1c10cad700 1 -- 172.21.15.19:0/3112339054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0c1074a0 msgr2=0x7f1c0c19c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:03.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.807+0000 7f1c10cad700 1 --2- 172.21.15.19:0/3112339054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0c1074a0 0x7f1c0c19c460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0117e0 tx=0x7f1bfc011ce0 comp rx=0 tx=0).stop 2024-04-25T11:57:03.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.807+0000 7f1c10cad700 1 -- 172.21.15.19:0/3112339054 shutdown_connections 2024-04-25T11:57:03.336 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.807+0000 7f1c10cad700 1 --2- 172.21.15.19:0/3112339054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c0c1074a0 0x7f1c0c19c460 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:03.337 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.807+0000 7f1c10cad700 1 -- 172.21.15.19:0/3112339054 >> 172.21.15.19:0/3112339054 conn(0x7f1c0c102a40 msgr2=0x7f1c0c103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:03.337 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.807+0000 7f1c10cad700 1 -- 172.21.15.19:0/3112339054 shutdown_connections 2024-04-25T11:57:03.337 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:02.807+0000 7f1c10cad700 1 -- 172.21.15.19:0/3112339054 wait complete. 2024-04-25T11:57:03.337 INFO:teuthology.orchestra.run.smithi019.stdout:Restarting the monitor... 2024-04-25T11:57:03.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:03 smithi019 ceph-mon[28843]: from='client.? 172.21.15.19:0/3112339054' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:57:03.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:03 smithi019 systemd[1]: Stopping Ceph mon.a for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T11:57:04.042 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:03 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a[28839]: 2024-04-25T11:57:03.618+0000 7f49dd4d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:57:04.042 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:03 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a[28839]: 2024-04-25T11:57:03.618+0000 7f49dd4d2700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T11:57:04.042 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:03 smithi019 podman[29089]: 2024-04-25 11:57:03.86462894 +0000 UTC m=+0.347382547 container died cbed7d58b21b88f295df90973547ff63d3117ca2b4c6a95f5fe4b42307109328 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T11:57:04.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:04 smithi019 podman[29089]: 2024-04-25 11:57:04.185134339 +0000 UTC m=+0.667887943 container cleanup cbed7d58b21b88f295df90973547ff63d3117ca2b4c6a95f5fe4b42307109328 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T11:57:04.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:04 smithi019 bash[29089]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a 2024-04-25T11:57:04.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:04 smithi019 podman[29103]: 2024-04-25 11:57:04.36087811 +0000 UTC m=+0.499714282 container remove cbed7d58b21b88f295df90973547ff63d3117ca2b4c6a95f5fe4b42307109328 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T11:57:04.928 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:04 smithi019 systemd[1]: ceph-be62e290-02fa-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T11:57:04.929 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:04 smithi019 systemd[1]: Stopped Ceph mon.a for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T11:57:04.929 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:04 smithi019 systemd[1]: Starting Ceph mon.a for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T11:57:04.929 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:04 smithi019 podman[29195]: 2024-04-25 11:57:04.738880083 +0000 UTC m=+0.018617024 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:57:04.929 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:04 smithi019 podman[29195]: 2024-04-25 11:57:04.853521752 +0000 UTC m=+0.133258708 container create 4a9fc10a4dcadd2afdfde1007aae6e54a191cc2eb2cfc031932e4fa418cce0fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T11:57:05.118 INFO:teuthology.orchestra.run.smithi019.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T11:57:05.232 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 podman[29195]: 2024-04-25 11:57:05.047931275 +0000 UTC m=+0.327668189 container init 4a9fc10a4dcadd2afdfde1007aae6e54a191cc2eb2cfc031932e4fa418cce0fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T11:57:05.232 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 podman[29195]: 2024-04-25 11:57:05.051087541 +0000 UTC m=+0.330824455 container start 4a9fc10a4dcadd2afdfde1007aae6e54a191cc2eb2cfc031932e4fa418cce0fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:57:05.232 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T11:57:05.232 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T11:57:05.232 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: pidfile_write: ignore empty --pid-file 2024-04-25T11:57:05.232 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: load: jerasure load: lrc 2024-04-25T11:57:05.232 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: RocksDB version: 7.9.2 2024-04-25T11:57:05.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Git sha 0 2024-04-25T11:57:05.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T11:57:05.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: DB SUMMARY 2024-04-25T11:57:05.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: DB Session ID: 76XV2SEXBY6OI7ZB4ZNS 2024-04-25T11:57:05.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: CURRENT file: CURRENT 2024-04-25T11:57:05.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T11:57:05.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T11:57:05.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T11:57:05.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105602 ; 2024-04-25T11:57:05.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.error_if_exists: 0 2024-04-25T11:57:05.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.create_if_missing: 0 2024-04-25T11:57:05.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.paranoid_checks: 1 2024-04-25T11:57:05.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T11:57:05.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T11:57:05.234 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T11:57:05.234 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.env: 0x55ef8e428cc0 2024-04-25T11:57:05.234 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T11:57:05.234 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.info_log: 0x55ef903b9be0 2024-04-25T11:57:05.234 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T11:57:05.234 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.statistics: (nil) 2024-04-25T11:57:05.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.use_fsync: 0 2024-04-25T11:57:05.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_log_file_size: 0 2024-04-25T11:57:05.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T11:57:05.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T11:57:05.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T11:57:05.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T11:57:05.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.allow_fallocate: 1 2024-04-25T11:57:05.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T11:57:05.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T11:57:05.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.use_direct_reads: 0 2024-04-25T11:57:05.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T11:57:05.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T11:57:05.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.db_log_dir: 2024-04-25T11:57:05.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.wal_dir: 2024-04-25T11:57:05.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T11:57:05.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T11:57:05.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T11:57:05.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T11:57:05.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T11:57:05.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T11:57:05.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T11:57:05.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T11:57:05.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.write_buffer_manager: 0x55ef9034d0e0 2024-04-25T11:57:05.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T11:57:05.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T11:57:05.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T11:57:05.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T11:57:05.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T11:57:05.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T11:57:05.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T11:57:05.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T11:57:05.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.unordered_write: 0 2024-04-25T11:57:05.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T11:57:05.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T11:57:05.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T11:57:05.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T11:57:05.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.row_cache: None 2024-04-25T11:57:05.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.wal_filter: None 2024-04-25T11:57:05.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T11:57:05.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T11:57:05.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.two_write_queues: 0 2024-04-25T11:57:05.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T11:57:05.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.wal_compression: 0 2024-04-25T11:57:05.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.atomic_flush: 0 2024-04-25T11:57:05.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T11:57:05.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T11:57:05.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T11:57:05.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.log_readahead_size: 0 2024-04-25T11:57:05.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T11:57:05.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T11:57:05.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T11:57:05.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T11:57:05.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T11:57:05.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T11:57:05.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T11:57:05.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_background_jobs: 2 2024-04-25T11:57:05.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_background_compactions: -1 2024-04-25T11:57:05.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_subcompactions: 1 2024-04-25T11:57:05.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T11:57:05.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T11:57:05.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T11:57:05.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T11:57:05.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T11:57:05.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T11:57:05.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T11:57:05.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T11:57:05.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_open_files: -1 2024-04-25T11:57:05.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T11:57:05.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T11:57:05.240 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T11:57:05.241 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T11:57:05.241 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_background_flushes: -1 2024-04-25T11:57:05.241 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Compression algorithms supported: 2024-04-25T11:57:05.241 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T11:57:05.241 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: kZSTD supported: 0 2024-04-25T11:57:05.241 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: kXpressCompression supported: 0 2024-04-25T11:57:05.241 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T11:57:05.241 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: kZlibCompression supported: 1 2024-04-25T11:57:05.241 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: kSnappyCompression supported: 1 2024-04-25T11:57:05.241 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: kLZ4Compression supported: 1 2024-04-25T11:57:05.242 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: kBZip2Compression supported: 0 2024-04-25T11:57:05.242 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T11:57:05.242 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T11:57:05.242 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T11:57:05.242 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T11:57:05.242 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T11:57:05.242 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.merge_operator: 2024-04-25T11:57:05.242 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compaction_filter: None 2024-04-25T11:57:05.242 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T11:57:05.242 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T11:57:05.242 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T11:57:05.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T11:57:05.243 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ef903b9dc0) 2024-04-25T11:57:05.243 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks: 1 2024-04-25T11:57:05.243 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T11:57:05.243 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T11:57:05.243 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_top_level_index_and_filter: 1 2024-04-25T11:57:05.243 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_type: 0 2024-04-25T11:57:05.243 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_index_type: 0 2024-04-25T11:57:05.243 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_shortening: 1 2024-04-25T11:57:05.243 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T11:57:05.243 INFO:journalctl@ceph.mon.a.smithi019.stdout: checksum: 4 2024-04-25T11:57:05.243 INFO:journalctl@ceph.mon.a.smithi019.stdout: no_block_cache: 0 2024-04-25T11:57:05.244 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache: 0x55ef9037f1f0 2024-04-25T11:57:05.244 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_name: BinnedLRUCache 2024-04-25T11:57:05.244 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_options: 2024-04-25T11:57:05.244 INFO:journalctl@ceph.mon.a.smithi019.stdout: capacity : 536870912 2024-04-25T11:57:05.244 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_shard_bits : 4 2024-04-25T11:57:05.244 INFO:journalctl@ceph.mon.a.smithi019.stdout: strict_capacity_limit : 0 2024-04-25T11:57:05.244 INFO:journalctl@ceph.mon.a.smithi019.stdout: high_pri_pool_ratio: 0.000 2024-04-25T11:57:05.244 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_compressed: (nil) 2024-04-25T11:57:05.244 INFO:journalctl@ceph.mon.a.smithi019.stdout: persistent_cache: (nil) 2024-04-25T11:57:05.244 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size: 4096 2024-04-25T11:57:05.244 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size_deviation: 10 2024-04-25T11:57:05.244 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_restart_interval: 16 2024-04-25T11:57:05.244 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_block_restart_interval: 1 2024-04-25T11:57:05.244 INFO:journalctl@ceph.mon.a.smithi019.stdout: metadata_block_size: 4096 2024-04-25T11:57:05.245 INFO:journalctl@ceph.mon.a.smithi019.stdout: partition_filters: 0 2024-04-25T11:57:05.245 INFO:journalctl@ceph.mon.a.smithi019.stdout: use_delta_encoding: 1 2024-04-25T11:57:05.245 INFO:journalctl@ceph.mon.a.smithi019.stdout: filter_policy: bloomfilter 2024-04-25T11:57:05.245 INFO:journalctl@ceph.mon.a.smithi019.stdout: whole_key_filtering: 1 2024-04-25T11:57:05.245 INFO:journalctl@ceph.mon.a.smithi019.stdout: verify_compression: 0 2024-04-25T11:57:05.245 INFO:journalctl@ceph.mon.a.smithi019.stdout: read_amp_bytes_per_bit: 0 2024-04-25T11:57:05.245 INFO:journalctl@ceph.mon.a.smithi019.stdout: format_version: 5 2024-04-25T11:57:05.245 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_index_compression: 1 2024-04-25T11:57:05.245 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_align: 0 2024-04-25T11:57:05.245 INFO:journalctl@ceph.mon.a.smithi019.stdout: max_auto_readahead_size: 262144 2024-04-25T11:57:05.245 INFO:journalctl@ceph.mon.a.smithi019.stdout: prepopulate_block_cache: 0 2024-04-25T11:57:05.245 INFO:journalctl@ceph.mon.a.smithi019.stdout: initial_auto_readahead_size: 8192 2024-04-25T11:57:05.245 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T11:57:05.245 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T11:57:05.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T11:57:05.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compression: NoCompression 2024-04-25T11:57:05.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T11:57:05.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T11:57:05.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T11:57:05.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.num_levels: 7 2024-04-25T11:57:05.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T11:57:05.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T11:57:05.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T11:57:05.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T11:57:05.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T11:57:05.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T11:57:05.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T11:57:05.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T11:57:05.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T11:57:05.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T11:57:05.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T11:57:05.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T11:57:05.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T11:57:05.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T11:57:05.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T11:57:05.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T11:57:05.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T11:57:05.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T11:57:05.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T11:57:05.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T11:57:05.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T11:57:05.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T11:57:05.248 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T11:57:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T11:57:05.250 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T11:57:05.250 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T11:57:05.250 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.inplace_update_support: 0 2024-04-25T11:57:05.250 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T11:57:05.250 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T11:57:05.250 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T11:57:05.250 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T11:57:05.250 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.bloom_locality: 0 2024-04-25T11:57:05.250 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.max_successive_merges: 0 2024-04-25T11:57:05.250 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T11:57:05.250 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T11:57:05.250 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T11:57:05.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T11:57:05.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.ttl: 2592000 2024-04-25T11:57:05.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T11:57:05.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T11:57:05.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T11:57:05.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.enable_blob_files: false 2024-04-25T11:57:05.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.min_blob_size: 0 2024-04-25T11:57:05.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T11:57:05.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T11:57:05.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T11:57:05.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T11:57:05.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T11:57:05.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T11:57:05.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T11:57:05.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T11:57:05.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T11:57:05.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T11:57:05.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 80e58c95-8dde-4786-b729-2b27d2968146 2024-04-25T11:57:05.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714046225080600, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T11:57:05.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T11:57:05.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714046225081692, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100689, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98573, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14288, "raw_average_key_size": 49, "raw_value_size": 90829, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714046225, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "80e58c95-8dde-4786-b729-2b27d2968146", "db_session_id": "76XV2SEXBY6OI7ZB4ZNS", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T11:57:05.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714046225081778, "job": 1, "event": "recovery_finished"} 2024-04-25T11:57:05.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T11:57:05.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T11:57:05.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ef90494000 2024-04-25T11:57:05.252 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: DB pointer 0x55ef9047e000 2024-04-25T11:57:05.253 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T11:57:05.253 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T11:57:05.253 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** DB Stats ** 2024-04-25T11:57:05.253 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T11:57:05.253 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T11:57:05.253 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T11:57:05.253 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T11:57:05.253 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T11:57:05.253 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T11:57:05.253 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T11:57:05.253 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T11:57:05.253 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-04-25T11:57:05.253 INFO:journalctl@ceph.mon.a.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T11:57:05.253 INFO:journalctl@ceph.mon.a.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T11:57:05.254 INFO:journalctl@ceph.mon.a.smithi019.stdout: L0 2/0 100.19 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T11:57:05.254 INFO:journalctl@ceph.mon.a.smithi019.stdout: Sum 2/0 100.19 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T11:57:05.254 INFO:journalctl@ceph.mon.a.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T11:57:05.254 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T11:57:05.254 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-04-25T11:57:05.254 INFO:journalctl@ceph.mon.a.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T11:57:05.254 INFO:journalctl@ceph.mon.a.smithi019.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T11:57:05.254 INFO:journalctl@ceph.mon.a.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T11:57:05.254 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T11:57:05.254 INFO:journalctl@ceph.mon.a.smithi019.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T11:57:05.254 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T11:57:05.254 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T11:57:05.254 INFO:journalctl@ceph.mon.a.smithi019.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T11:57:05.255 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T11:57:05.255 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T11:57:05.255 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T11:57:05.255 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T11:57:05.255 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative compaction: 0.00 GB write, 30.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T11:57:05.255 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval compaction: 0.00 GB write, 30.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T11:57:05.255 INFO:journalctl@ceph.mon.a.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T11:57:05.255 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache BinnedLRUCache@0x55ef9037f1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T11:57:05.255 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T11:57:05.255 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T11:57:05.255 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T11:57:05.255 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: starting mon.a rank 0 at public addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] at bind addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:57:05.255 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: mon.a@-1(???) e1 preinit fsid be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:57:05.255 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: mon.a@-1(???).mds e1 new map 2024-04-25T11:57:05.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: mon.a@-1(???).mds e1 print_map 2024-04-25T11:57:05.256 INFO:journalctl@ceph.mon.a.smithi019.stdout: e1 2024-04-25T11:57:05.256 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T11:57:05.256 INFO:journalctl@ceph.mon.a.smithi019.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T11:57:05.256 INFO:journalctl@ceph.mon.a.smithi019.stdout: legacy client fscid: -1 2024-04-25T11:57:05.256 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-04-25T11:57:05.256 INFO:journalctl@ceph.mon.a.smithi019.stdout: No filesystems configured 2024-04-25T11:57:05.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T11:57:05.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T11:57:05.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T11:57:05.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T11:57:05.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T11:57:05.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 bash[29195]: 4a9fc10a4dcadd2afdfde1007aae6e54a191cc2eb2cfc031932e4fa418cce0fa 2024-04-25T11:57:05.257 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 systemd[1]: Started Ceph mon.a for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T11:57:05.257 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T11:57:05.257 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: monmap epoch 1 2024-04-25T11:57:05.257 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: fsid be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:57:05.257 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: last_changed 2024-04-25T11:56:56.493140+0000 2024-04-25T11:57:05.257 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: created 2024-04-25T11:56:56.493140+0000 2024-04-25T11:57:05.257 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: min_mon_release 19 (squid) 2024-04-25T11:57:05.257 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: election_strategy: 1 2024-04-25T11:57:05.257 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T11:57:05.257 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: fsmap 2024-04-25T11:57:05.257 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T11:57:05.257 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:05 smithi019 ceph-mon[29213]: mgrmap e1: no daemons active 2024-04-25T11:57:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.639+0000 7f05fc3cb700 1 Processor -- start 2024-04-25T11:57:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.639+0000 7f05fc3cb700 1 -- start start 2024-04-25T11:57:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.640+0000 7f05fc3cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f407de40 0x7f05f407c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.640+0000 7f05fc3cb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05f407c8c0 con 0x7f05f407de40 2024-04-25T11:57:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.640+0000 7f05fa167700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f407de40 0x7f05f407c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.640+0000 7f05fa167700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f407de40 0x7f05f407c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44716/0 (socket says 172.21.15.19:44716) 2024-04-25T11:57:06.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.640+0000 7f05fa167700 1 -- 172.21.15.19:0/1117235108 learned_addr learned my addr 172.21.15.19:0/1117235108 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:06.268 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.640+0000 7f05fa167700 1 -- 172.21.15.19:0/1117235108 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f407ca40 con 0x7f05f407de40 2024-04-25T11:57:06.268 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.641+0000 7f05fa167700 1 --2- 172.21.15.19:0/1117235108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f407de40 0x7f05f407c2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f05f00098a0 tx=0x7f05f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf58c1f1bae9b76b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:06.268 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.641+0000 7f05f9165700 1 -- 172.21.15.19:0/1117235108 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f0004160 con 0x7f05f407de40 2024-04-25T11:57:06.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.641+0000 7f05f9165700 1 -- 172.21.15.19:0/1117235108 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f05f0004300 con 0x7f05f407de40 2024-04-25T11:57:06.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.641+0000 7f05f9165700 1 -- 172.21.15.19:0/1117235108 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f0011b20 con 0x7f05f407de40 2024-04-25T11:57:06.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.641+0000 7f05fc3cb700 1 -- 172.21.15.19:0/1117235108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f407de40 msgr2=0x7f05f407c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:06.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.641+0000 7f05fc3cb700 1 --2- 172.21.15.19:0/1117235108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f407de40 0x7f05f407c2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f05f00098a0 tx=0x7f05f0009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:06.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.641+0000 7f05fc3cb700 1 -- 172.21.15.19:0/1117235108 shutdown_connections 2024-04-25T11:57:06.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.641+0000 7f05fc3cb700 1 --2- 172.21.15.19:0/1117235108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f407de40 0x7f05f407c2f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:06.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.641+0000 7f05fc3cb700 1 -- 172.21.15.19:0/1117235108 >> 172.21.15.19:0/1117235108 conn(0x7f05f4103bb0 msgr2=0x7f05f4105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:06.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.642+0000 7f05fc3cb700 1 -- 172.21.15.19:0/1117235108 shutdown_connections 2024-04-25T11:57:06.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.642+0000 7f05fc3cb700 1 -- 172.21.15.19:0/1117235108 wait complete. 2024-04-25T11:57:06.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.642+0000 7f05fc3cb700 1 Processor -- start 2024-04-25T11:57:06.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.642+0000 7f05fc3cb700 1 -- start start 2024-04-25T11:57:06.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.642+0000 7f05fc3cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f407de40 0x7f05f419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:06.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.642+0000 7f05fc3cb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05f410eb60 con 0x7f05f407de40 2024-04-25T11:57:06.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.642+0000 7f05fa167700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f407de40 0x7f05f419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:06.273 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.642+0000 7f05fa167700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f407de40 0x7f05f419e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44732/0 (socket says 172.21.15.19:44732) 2024-04-25T11:57:06.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.642+0000 7f05fa167700 1 -- 172.21.15.19:0/3213235293 learned_addr learned my addr 172.21.15.19:0/3213235293 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:06.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.643+0000 7f05fa167700 1 -- 172.21.15.19:0/3213235293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f419eb40 con 0x7f05f407de40 2024-04-25T11:57:06.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.643+0000 7f05fa167700 1 --2- 172.21.15.19:0/3213235293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f407de40 0x7f05f419e600 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f05f00116a0 tx=0x7f05f0011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:06.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.643+0000 7f05eb7fe700 1 -- 172.21.15.19:0/3213235293 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f0003ab0 con 0x7f05f407de40 2024-04-25T11:57:06.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.643+0000 7f05fc3cb700 1 -- 172.21.15.19:0/3213235293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f419edd0 con 0x7f05f407de40 2024-04-25T11:57:06.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.643+0000 7f05eb7fe700 1 -- 172.21.15.19:0/3213235293 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f05f0003c50 con 0x7f05f407de40 2024-04-25T11:57:06.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.643+0000 7f05eb7fe700 1 -- 172.21.15.19:0/3213235293 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f0019870 con 0x7f05f407de40 2024-04-25T11:57:06.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.643+0000 7f05fc3cb700 1 -- 172.21.15.19:0/3213235293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f419f230 con 0x7f05f407de40 2024-04-25T11:57:06.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.643+0000 7f05eb7fe700 1 -- 172.21.15.19:0/3213235293 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f05f002d430 con 0x7f05f407de40 2024-04-25T11:57:06.274 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.643+0000 7f05eb7fe700 1 -- 172.21.15.19:0/3213235293 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f05f00247d0 con 0x7f05f407de40 2024-04-25T11:57:06.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.644+0000 7f05fc3cb700 1 -- 172.21.15.19:0/3213235293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f40640a0 con 0x7f05f407de40 2024-04-25T11:57:06.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.647+0000 7f05eb7fe700 1 -- 172.21.15.19:0/3213235293 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f05f0011e20 con 0x7f05f407de40 2024-04-25T11:57:06.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.718+0000 7f05fc3cb700 1 -- 172.21.15.19:0/3213235293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f05f41a1ca0 con 0x7f05f407de40 2024-04-25T11:57:06.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.720+0000 7f05eb7fe700 1 -- 172.21.15.19:0/3213235293 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f05f001ac00 con 0x7f05f407de40 2024-04-25T11:57:06.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.721+0000 7f05fc3cb700 1 -- 172.21.15.19:0/3213235293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f407de40 msgr2=0x7f05f419e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:06.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.721+0000 7f05fc3cb700 1 --2- 172.21.15.19:0/3213235293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f407de40 0x7f05f419e600 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f05f00116a0 tx=0x7f05f0011ce0 comp rx=0 tx=0).stop 2024-04-25T11:57:06.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.721+0000 7f05fc3cb700 1 -- 172.21.15.19:0/3213235293 shutdown_connections 2024-04-25T11:57:06.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.721+0000 7f05fc3cb700 1 --2- 172.21.15.19:0/3213235293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05f407de40 0x7f05f419e600 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:06.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.721+0000 7f05fc3cb700 1 -- 172.21.15.19:0/3213235293 >> 172.21.15.19:0/3213235293 conn(0x7f05f4103bb0 msgr2=0x7f05f4104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:06.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.721+0000 7f05fc3cb700 1 -- 172.21.15.19:0/3213235293 shutdown_connections 2024-04-25T11:57:06.275 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:05.721+0000 7f05fc3cb700 1 -- 172.21.15.19:0/3213235293 wait complete. 2024-04-25T11:57:06.275 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T11:57:06.297 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:57:06.297 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mgr... 2024-04-25T11:57:06.298 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T11:57:06.298 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T11:57:06.547 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-be62e290-02fa-11ef-bc93-c7b262605968@mgr.x 2024-04-25T11:57:06.547 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-be62e290-02fa-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-be62e290-02fa-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T11:57:06.697 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-be62e290-02fa-11ef-bc93-c7b262605968.target.wants/ceph-be62e290-02fa-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-be62e290-02fa-11ef-bc93-c7b262605968@.service. 2024-04-25T11:57:07.024 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:06 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3213235293' entity='client.admin' 2024-04-25T11:57:07.231 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T11:57:07.231 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-25T11:57:07.239 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T11:57:07.239 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T11:57:07.239 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-25T11:57:07.239 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T11:57:07.248 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T11:57:07.248 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-25T11:57:07.256 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T11:57:07.256 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T11:57:07.256 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-25T11:57:07.256 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T11:57:07.256 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr to start... 2024-04-25T11:57:07.256 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr... 2024-04-25T11:57:08.025 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:07 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1796655754' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T11:57:08.026 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:07 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:07.943+0000 7f659ec7d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:57:08.026 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:08 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:08.033+0000 7f659ec7d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:57:08.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T11:57:08.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:57:08.419 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "be62e290-02fa-11ef-bc93-c7b262605968", 2024-04-25T11:57:08.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T11:57:08.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T11:57:08.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T11:57:08.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T11:57:08.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:08.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T11:57:08.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T11:57:08.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-04-25T11:57:08.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:57:08.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T11:57:08.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T11:57:08.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:57:08.420 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T11:57:08.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T11:57:08.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:08.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T11:57:08.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T11:57:08.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:08.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T11:57:08.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:08.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T11:57:08.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T11:57:08.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T11:57:08.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T11:57:08.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T11:57:08.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T11:57:08.421 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:08.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T11:57:08.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T11:57:08.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T11:57:08.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T11:57:08.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T11:57:08.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T11:57:08.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T11:57:08.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T11:57:08.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T11:57:08.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:08.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T11:57:08.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:08.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T11:57:08.422 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T11:57:00.082903+0000", 2024-04-25T11:57:08.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:57:08.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:08.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T11:57:08.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:57:08.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.731+0000 7f864b538700 1 Processor -- start 2024-04-25T11:57:08.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.731+0000 7f864b538700 1 -- start start 2024-04-25T11:57:08.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.731+0000 7f864b538700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8644109090 0x7f864410b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:08.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.731+0000 7f864b538700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f864407be20 con 0x7f8644109090 2024-04-25T11:57:08.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.732+0000 7f86492d4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8644109090 0x7f864410b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:08.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.732+0000 7f86492d4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8644109090 0x7f864410b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44754/0 (socket says 172.21.15.19:44754) 2024-04-25T11:57:08.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.732+0000 7f86492d4700 1 -- 172.21.15.19:0/2086138277 learned_addr learned my addr 172.21.15.19:0/2086138277 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:08.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.732+0000 7f86492d4700 1 -- 172.21.15.19:0/2086138277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864410b9c0 con 0x7f8644109090 2024-04-25T11:57:08.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.732+0000 7f86492d4700 1 --2- 172.21.15.19:0/2086138277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8644109090 0x7f864410b480 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8634009e80 tx=0x7f863400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5209256764ca212a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:08.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.732+0000 7f8643fff700 1 -- 172.21.15.19:0/2086138277 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8634017070 con 0x7f8644109090 2024-04-25T11:57:08.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.733+0000 7f8643fff700 1 -- 172.21.15.19:0/2086138277 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f863400b8c0 con 0x7f8644109090 2024-04-25T11:57:08.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.733+0000 7f8643fff700 1 -- 172.21.15.19:0/2086138277 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86340045c0 con 0x7f8644109090 2024-04-25T11:57:08.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.733+0000 7f864b538700 1 -- 172.21.15.19:0/2086138277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8644109090 msgr2=0x7f864410b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:08.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.733+0000 7f864b538700 1 --2- 172.21.15.19:0/2086138277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8644109090 0x7f864410b480 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8634009e80 tx=0x7f863400b330 comp rx=0 tx=0).stop 2024-04-25T11:57:08.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.733+0000 7f864b538700 1 -- 172.21.15.19:0/2086138277 shutdown_connections 2024-04-25T11:57:08.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.733+0000 7f864b538700 1 --2- 172.21.15.19:0/2086138277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8644109090 0x7f864410b480 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:08.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.733+0000 7f864b538700 1 -- 172.21.15.19:0/2086138277 >> 172.21.15.19:0/2086138277 conn(0x7f8644102a40 msgr2=0x7f8644104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:08.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.734+0000 7f864b538700 1 -- 172.21.15.19:0/2086138277 shutdown_connections 2024-04-25T11:57:08.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.734+0000 7f864b538700 1 -- 172.21.15.19:0/2086138277 wait complete. 2024-04-25T11:57:08.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.734+0000 7f864b538700 1 Processor -- start 2024-04-25T11:57:08.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.734+0000 7f864b538700 1 -- start start 2024-04-25T11:57:08.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.734+0000 7f864b538700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8644109090 0x7f864419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:08.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.734+0000 7f864b538700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f864410c980 con 0x7f8644109090 2024-04-25T11:57:08.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.734+0000 7f86492d4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8644109090 0x7f864419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:08.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.735+0000 7f86492d4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8644109090 0x7f864419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44760/0 (socket says 172.21.15.19:44760) 2024-04-25T11:57:08.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.735+0000 7f86492d4700 1 -- 172.21.15.19:0/1796655754 learned_addr learned my addr 172.21.15.19:0/1796655754 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:08.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.735+0000 7f86492d4700 1 -- 172.21.15.19:0/1796655754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864419c9a0 con 0x7f8644109090 2024-04-25T11:57:08.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.735+0000 7f86492d4700 1 --2- 172.21.15.19:0/1796655754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8644109090 0x7f864419c460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f863400b7b0 tx=0x7f863400bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:08.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.735+0000 7f86427fc700 1 -- 172.21.15.19:0/1796655754 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8634017070 con 0x7f8644109090 2024-04-25T11:57:08.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.735+0000 7f86427fc700 1 -- 172.21.15.19:0/1796655754 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f8634022070 con 0x7f8644109090 2024-04-25T11:57:08.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.735+0000 7f864b538700 1 -- 172.21.15.19:0/1796655754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864419cc30 con 0x7f8644109090 2024-04-25T11:57:08.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.735+0000 7f86427fc700 1 -- 172.21.15.19:0/1796655754 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f863401d6d0 con 0x7f8644109090 2024-04-25T11:57:08.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.735+0000 7f864b538700 1 -- 172.21.15.19:0/1796655754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864419f920 con 0x7f8644109090 2024-04-25T11:57:08.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.735+0000 7f86427fc700 1 -- 172.21.15.19:0/1796655754 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f863401d980 con 0x7f8644109090 2024-04-25T11:57:08.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.736+0000 7f86427fc700 1 -- 172.21.15.19:0/1796655754 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f863400f050 con 0x7f8644109090 2024-04-25T11:57:08.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.736+0000 7f864b538700 1 -- 172.21.15.19:0/1796655754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86440640a0 con 0x7f8644109090 2024-04-25T11:57:08.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.739+0000 7f86427fc700 1 -- 172.21.15.19:0/1796655754 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8634003d60 con 0x7f8644109090 2024-04-25T11:57:08.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.813+0000 7f864b538700 1 -- 172.21.15.19:0/1796655754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f864419faf0 con 0x7f8644109090 2024-04-25T11:57:08.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.813+0000 7f86427fc700 1 -- 172.21.15.19:0/1796655754 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8634027460 con 0x7f8644109090 2024-04-25T11:57:08.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.814+0000 7f864b538700 1 -- 172.21.15.19:0/1796655754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8644109090 msgr2=0x7f864419c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:08.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.814+0000 7f864b538700 1 --2- 172.21.15.19:0/1796655754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8644109090 0x7f864419c460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f863400b7b0 tx=0x7f863400bdc0 comp rx=0 tx=0).stop 2024-04-25T11:57:08.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.814+0000 7f864b538700 1 -- 172.21.15.19:0/1796655754 shutdown_connections 2024-04-25T11:57:08.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.814+0000 7f864b538700 1 --2- 172.21.15.19:0/1796655754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8644109090 0x7f864419c460 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:08.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.814+0000 7f864b538700 1 -- 172.21.15.19:0/1796655754 >> 172.21.15.19:0/1796655754 conn(0x7f8644102a40 msgr2=0x7f86441034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:08.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.814+0000 7f864b538700 1 -- 172.21.15.19:0/1796655754 shutdown_connections 2024-04-25T11:57:08.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:07.814+0000 7f864b538700 1 -- 172.21.15.19:0/1796655754 wait complete. 2024-04-25T11:57:08.427 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (1/15)... 2024-04-25T11:57:09.648 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:09.305+0000 7f659ec7d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:57:09.649 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:09.481+0000 7f659ec7d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:57:09.649 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:09.657+0000 7f659ec7d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:57:10.256 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:09.957+0000 7f659ec7d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:57:10.256 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:10 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:10.054+0000 7f659ec7d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:57:10.256 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:10 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:10.264+0000 7f659ec7d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:57:10.541 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:10 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:10.483+0000 7f659ec7d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:57:11.118 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:10 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3079984291' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T11:57:11.529 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:11 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:11.126+0000 7f659ec7d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:57:11.536 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T11:57:11.536 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:57:11.536 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "be62e290-02fa-11ef-bc93-c7b262605968", 2024-04-25T11:57:11.536 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T11:57:11.536 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T11:57:11.536 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T11:57:11.536 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T11:57:11.536 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:11.536 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T11:57:11.536 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T11:57:11.537 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-04-25T11:57:11.537 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:57:11.537 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T11:57:11.537 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T11:57:11.537 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:57:11.537 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T11:57:11.537 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T11:57:11.537 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:11.537 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T11:57:11.537 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T11:57:11.538 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:11.538 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T11:57:11.538 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:11.538 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T11:57:11.538 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T11:57:11.538 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T11:57:11.538 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T11:57:11.539 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T11:57:11.539 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T11:57:11.539 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:11.539 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T11:57:11.539 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T11:57:11.548 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T11:57:11.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T11:57:11.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T11:57:11.549 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T11:57:11.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T11:57:11.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T11:57:11.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T11:57:11.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:11.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T11:57:11.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:11.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T11:57:11.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T11:57:11.550 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:11.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T11:57:11.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T11:57:11.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T11:57:11.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T11:57:11.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T11:57:11.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T11:57:11.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T11:57:11.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:57:11.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:57:11.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:11.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T11:57:11.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:11.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T11:57:00.082903+0000", 2024-04-25T11:57:11.551 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:57:11.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:11.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T11:57:11.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:57:11.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.852+0000 7fbca1492700 1 Processor -- start 2024-04-25T11:57:11.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.852+0000 7fbca1492700 1 -- start start 2024-04-25T11:57:11.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.852+0000 7fbca1492700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc9c109070 0x7fbc9c10b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:11.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.852+0000 7fbca1492700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c07be60 con 0x7fbc9c109070 2024-04-25T11:57:11.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.853+0000 7fbc9affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc9c109070 0x7fbc9c10b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:11.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.853+0000 7fbc9affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc9c109070 0x7fbc9c10b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44774/0 (socket says 172.21.15.19:44774) 2024-04-25T11:57:11.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.853+0000 7fbc9affd700 1 -- 172.21.15.19:0/54231135 learned_addr learned my addr 172.21.15.19:0/54231135 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:11.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.853+0000 7fbc9affd700 1 -- 172.21.15.19:0/54231135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9c10b9a0 con 0x7fbc9c109070 2024-04-25T11:57:11.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.853+0000 7fbc9affd700 1 --2- 172.21.15.19:0/54231135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc9c109070 0x7fbc9c10b460 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc900098a0 tx=0x7fbc90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f89e2038c857e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:11.552 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.853+0000 7fbc99ffb700 1 -- 172.21.15.19:0/54231135 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc90004160 con 0x7fbc9c109070 2024-04-25T11:57:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.853+0000 7fbc99ffb700 1 -- 172.21.15.19:0/54231135 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fbc90004300 con 0x7fbc9c109070 2024-04-25T11:57:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.853+0000 7fbc99ffb700 1 -- 172.21.15.19:0/54231135 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc90011b20 con 0x7fbc9c109070 2024-04-25T11:57:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.854+0000 7fbca1492700 1 -- 172.21.15.19:0/54231135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc9c109070 msgr2=0x7fbc9c10b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.854+0000 7fbca1492700 1 --2- 172.21.15.19:0/54231135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc9c109070 0x7fbc9c10b460 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc900098a0 tx=0x7fbc90009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.854+0000 7fbca1492700 1 -- 172.21.15.19:0/54231135 shutdown_connections 2024-04-25T11:57:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.854+0000 7fbca1492700 1 --2- 172.21.15.19:0/54231135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc9c109070 0x7fbc9c10b460 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.854+0000 7fbca1492700 1 -- 172.21.15.19:0/54231135 >> 172.21.15.19:0/54231135 conn(0x7fbc9c102a80 msgr2=0x7fbc9c104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.854+0000 7fbca1492700 1 -- 172.21.15.19:0/54231135 shutdown_connections 2024-04-25T11:57:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.854+0000 7fbca1492700 1 -- 172.21.15.19:0/54231135 wait complete. 2024-04-25T11:57:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.855+0000 7fbca1492700 1 Processor -- start 2024-04-25T11:57:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.855+0000 7fbca1492700 1 -- start start 2024-04-25T11:57:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.855+0000 7fbca1492700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc9c109070 0x7fbc9c1088f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.855+0000 7fbca1492700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c10c9c0 con 0x7fbc9c109070 2024-04-25T11:57:11.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.855+0000 7fbc9affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc9c109070 0x7fbc9c1088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:11.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.855+0000 7fbc9affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc9c109070 0x7fbc9c1088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44780/0 (socket says 172.21.15.19:44780) 2024-04-25T11:57:11.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.855+0000 7fbc9affd700 1 -- 172.21.15.19:0/3079984291 learned_addr learned my addr 172.21.15.19:0/3079984291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:11.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.855+0000 7fbc9affd700 1 -- 172.21.15.19:0/3079984291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9c108e30 con 0x7fbc9c109070 2024-04-25T11:57:11.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.856+0000 7fbc9affd700 1 --2- 172.21.15.19:0/3079984291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc9c109070 0x7fbc9c1088f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbc90006af0 tx=0x7fbc90011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:11.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.856+0000 7fbc83fff700 1 -- 172.21.15.19:0/3079984291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc900039a0 con 0x7fbc9c109070 2024-04-25T11:57:11.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.856+0000 7fbc83fff700 1 -- 172.21.15.19:0/3079984291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fbc90003b40 con 0x7fbc9c109070 2024-04-25T11:57:11.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.856+0000 7fbca1492700 1 -- 172.21.15.19:0/3079984291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9c107020 con 0x7fbc9c109070 2024-04-25T11:57:11.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.856+0000 7fbc83fff700 1 -- 172.21.15.19:0/3079984291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc90019670 con 0x7fbc9c109070 2024-04-25T11:57:11.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.856+0000 7fbca1492700 1 -- 172.21.15.19:0/3079984291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9c1074e0 con 0x7fbc9c109070 2024-04-25T11:57:11.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.856+0000 7fbc83fff700 1 -- 172.21.15.19:0/3079984291 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbc90019c90 con 0x7fbc9c109070 2024-04-25T11:57:11.554 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.856+0000 7fbc83fff700 1 -- 172.21.15.19:0/3079984291 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbc90024730 con 0x7fbc9c109070 2024-04-25T11:57:11.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.857+0000 7fbca1492700 1 -- 172.21.15.19:0/3079984291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc9c0640a0 con 0x7fbc9c109070 2024-04-25T11:57:11.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.861+0000 7fbc83fff700 1 -- 172.21.15.19:0/3079984291 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbc90043e50 con 0x7fbc9c109070 2024-04-25T11:57:11.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.935+0000 7fbca1492700 1 -- 172.21.15.19:0/3079984291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbc9c107a10 con 0x7fbc9c109070 2024-04-25T11:57:11.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.935+0000 7fbc83fff700 1 -- 172.21.15.19:0/3079984291 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fbc90032020 con 0x7fbc9c109070 2024-04-25T11:57:11.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.936+0000 7fbca1492700 1 -- 172.21.15.19:0/3079984291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc9c109070 msgr2=0x7fbc9c1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:11.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.936+0000 7fbca1492700 1 --2- 172.21.15.19:0/3079984291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc9c109070 0x7fbc9c1088f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbc90006af0 tx=0x7fbc90011ce0 comp rx=0 tx=0).stop 2024-04-25T11:57:11.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.937+0000 7fbca1492700 1 -- 172.21.15.19:0/3079984291 shutdown_connections 2024-04-25T11:57:11.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.937+0000 7fbca1492700 1 --2- 172.21.15.19:0/3079984291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc9c109070 0x7fbc9c1088f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:11.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.937+0000 7fbca1492700 1 -- 172.21.15.19:0/3079984291 >> 172.21.15.19:0/3079984291 conn(0x7fbc9c102a80 msgr2=0x7fbc9c103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:11.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.937+0000 7fbca1492700 1 -- 172.21.15.19:0/3079984291 shutdown_connections 2024-04-25T11:57:11.555 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:10.937+0000 7fbca1492700 1 -- 172.21.15.19:0/3079984291 wait complete. 2024-04-25T11:57:11.556 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (2/15)... 2024-04-25T11:57:11.792 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:11 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:11.569+0000 7f659ec7d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:57:11.792 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:11 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:11.669+0000 7f659ec7d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:57:12.291 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:11 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:11.877+0000 7f659ec7d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:57:12.568 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:12 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:12.428+0000 7f659ec7d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:57:12.569 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:12 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:12.525+0000 7f659ec7d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:57:12.921 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:12 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:12.787+0000 7f659ec7d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:57:12.921 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:12 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:12.929+0000 7f659ec7d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:57:13.291 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:13 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:13.030+0000 7f659ec7d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:57:13.291 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:13 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:13.152+0000 7f659ec7d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:57:13.756 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:13 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:13.467+0000 7f659ec7d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:57:14.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:14 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3276256119' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T11:57:14.364 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:14 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:14.143+0000 7f659ec7d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:57:14.364 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:14 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:14.372+0000 7f659ec7d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:57:14.857 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T11:57:14.857 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:57:14.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "be62e290-02fa-11ef-bc93-c7b262605968", 2024-04-25T11:57:14.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T11:57:14.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T11:57:14.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T11:57:14.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T11:57:14.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:14.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T11:57:14.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T11:57:14.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-04-25T11:57:14.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:57:14.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T11:57:14.858 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T11:57:14.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:57:14.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-25T11:57:14.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T11:57:14.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:14.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T11:57:14.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T11:57:14.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:14.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T11:57:14.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:14.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T11:57:14.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T11:57:14.859 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T11:57:14.867 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T11:57:14.867 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T11:57:14.867 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T11:57:14.867 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:14.867 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T11:57:14.867 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T11:57:14.867 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T11:57:14.867 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T11:57:14.867 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T11:57:14.867 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T11:57:14.868 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T11:57:14.868 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T11:57:14.868 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T11:57:14.868 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:14.868 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T11:57:14.868 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:14.868 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T11:57:14.868 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T11:57:14.868 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:14.868 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T11:57:14.868 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T11:57:14.868 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T11:57:14.868 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T11:57:00.082903+0000", 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.968+0000 7fbcb286f700 1 Processor -- start 2024-04-25T11:57:14.869 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.968+0000 7fbcb286f700 1 -- start start 2024-04-25T11:57:14.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.968+0000 7fbcb286f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcac106e80 0x7fbcac1072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:14.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.968+0000 7fbcb286f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac07be20 con 0x7fbcac106e80 2024-04-25T11:57:14.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.969+0000 7fbcabfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcac106e80 0x7fbcac1072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:14.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.969+0000 7fbcabfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcac106e80 0x7fbcac1072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58440/0 (socket says 172.21.15.19:58440) 2024-04-25T11:57:14.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.969+0000 7fbcabfff700 1 -- 172.21.15.19:0/755049613 learned_addr learned my addr 172.21.15.19:0/755049613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:14.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.969+0000 7fbcabfff700 1 -- 172.21.15.19:0/755049613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcac1077e0 con 0x7fbcac106e80 2024-04-25T11:57:14.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.969+0000 7fbcabfff700 1 --2- 172.21.15.19:0/755049613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcac106e80 0x7fbcac1072a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc940098a0 tx=0x7fbc94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3fc027bdc41f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:14.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.969+0000 7fbcaaffd700 1 -- 172.21.15.19:0/755049613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc94004160 con 0x7fbcac106e80 2024-04-25T11:57:14.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.970+0000 7fbcaaffd700 1 -- 172.21.15.19:0/755049613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fbc94004300 con 0x7fbcac106e80 2024-04-25T11:57:14.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.970+0000 7fbcaaffd700 1 -- 172.21.15.19:0/755049613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc94011b20 con 0x7fbcac106e80 2024-04-25T11:57:14.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.970+0000 7fbcb286f700 1 -- 172.21.15.19:0/755049613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcac106e80 msgr2=0x7fbcac1072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:14.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.970+0000 7fbcb286f700 1 --2- 172.21.15.19:0/755049613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcac106e80 0x7fbcac1072a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc940098a0 tx=0x7fbc94009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:14.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.970+0000 7fbcb286f700 1 -- 172.21.15.19:0/755049613 shutdown_connections 2024-04-25T11:57:14.870 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.970+0000 7fbcb286f700 1 --2- 172.21.15.19:0/755049613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcac106e80 0x7fbcac1072a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:14.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.970+0000 7fbcb286f700 1 -- 172.21.15.19:0/755049613 >> 172.21.15.19:0/755049613 conn(0x7fbcac102a40 msgr2=0x7fbcac104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:14.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.970+0000 7fbcb286f700 1 -- 172.21.15.19:0/755049613 shutdown_connections 2024-04-25T11:57:14.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.970+0000 7fbcb286f700 1 -- 172.21.15.19:0/755049613 wait complete. 2024-04-25T11:57:14.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.971+0000 7fbcb286f700 1 Processor -- start 2024-04-25T11:57:14.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.971+0000 7fbcb286f700 1 -- start start 2024-04-25T11:57:14.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.971+0000 7fbcb286f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcac106e80 0x7fbcac19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:14.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.971+0000 7fbcb286f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac108870 con 0x7fbcac106e80 2024-04-25T11:57:14.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.972+0000 7fbcabfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcac106e80 0x7fbcac19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:14.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.972+0000 7fbcabfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcac106e80 0x7fbcac19e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58444/0 (socket says 172.21.15.19:58444) 2024-04-25T11:57:14.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.972+0000 7fbcabfff700 1 -- 172.21.15.19:0/3276256119 learned_addr learned my addr 172.21.15.19:0/3276256119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:14.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.972+0000 7fbcabfff700 1 -- 172.21.15.19:0/3276256119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcac19eb50 con 0x7fbcac106e80 2024-04-25T11:57:14.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.972+0000 7fbcabfff700 1 --2- 172.21.15.19:0/3276256119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcac106e80 0x7fbcac19e610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc94006af0 tx=0x7fbc94011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:14.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.972+0000 7fbca97fa700 1 -- 172.21.15.19:0/3276256119 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc940039a0 con 0x7fbcac106e80 2024-04-25T11:57:14.871 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.972+0000 7fbca97fa700 1 -- 172.21.15.19:0/3276256119 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fbc94003b40 con 0x7fbcac106e80 2024-04-25T11:57:14.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.972+0000 7fbcb286f700 1 -- 172.21.15.19:0/3276256119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcac19ede0 con 0x7fbcac106e80 2024-04-25T11:57:14.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.972+0000 7fbca97fa700 1 -- 172.21.15.19:0/3276256119 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc94019670 con 0x7fbcac106e80 2024-04-25T11:57:14.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.972+0000 7fbcb286f700 1 -- 172.21.15.19:0/3276256119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcac19f240 con 0x7fbcac106e80 2024-04-25T11:57:14.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.972+0000 7fbca97fa700 1 -- 172.21.15.19:0/3276256119 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbc94019d00 con 0x7fbcac106e80 2024-04-25T11:57:14.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.973+0000 7fbca97fa700 1 -- 172.21.15.19:0/3276256119 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbc94024780 con 0x7fbcac106e80 2024-04-25T11:57:14.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.973+0000 7fbcb286f700 1 -- 172.21.15.19:0/3276256119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcac0640a0 con 0x7fbcac106e80 2024-04-25T11:57:14.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:13.976+0000 7fbca97fa700 1 -- 172.21.15.19:0/3276256119 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbc94043e50 con 0x7fbcac106e80 2024-04-25T11:57:14.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:14.072+0000 7fbcb286f700 1 -- 172.21.15.19:0/3276256119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbcac02df80 con 0x7fbcac106e80 2024-04-25T11:57:14.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:14.073+0000 7fbca97fa700 1 -- 172.21.15.19:0/3276256119 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fbc94032020 con 0x7fbcac106e80 2024-04-25T11:57:14.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:14.074+0000 7fbcb286f700 1 -- 172.21.15.19:0/3276256119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcac106e80 msgr2=0x7fbcac19e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:14.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:14.074+0000 7fbcb286f700 1 --2- 172.21.15.19:0/3276256119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcac106e80 0x7fbcac19e610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc94006af0 tx=0x7fbc94011ce0 comp rx=0 tx=0).stop 2024-04-25T11:57:14.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:14.074+0000 7fbcb286f700 1 -- 172.21.15.19:0/3276256119 shutdown_connections 2024-04-25T11:57:14.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:14.074+0000 7fbcb286f700 1 --2- 172.21.15.19:0/3276256119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcac106e80 0x7fbcac19e610 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:14.872 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:14.074+0000 7fbcb286f700 1 -- 172.21.15.19:0/3276256119 >> 172.21.15.19:0/3276256119 conn(0x7fbcac102a40 msgr2=0x7fbcac1952a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:14.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:14.074+0000 7fbcb286f700 1 -- 172.21.15.19:0/3276256119 shutdown_connections 2024-04-25T11:57:14.873 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:14.074+0000 7fbcb286f700 1 -- 172.21.15.19:0/3276256119 wait complete. 2024-04-25T11:57:14.873 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (3/15)... 2024-04-25T11:57:15.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:15 smithi019 ceph-mon[29213]: Activating manager daemon x 2024-04-25T11:57:15.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:15 smithi019 ceph-mon[29213]: mgrmap e2: x(active, starting, since 0.00349242s) 2024-04-25T11:57:15.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:15 smithi019 ceph-mon[29213]: from='mgr.14100 172.21.15.19:0/3985619028' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T11:57:15.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:15 smithi019 ceph-mon[29213]: from='mgr.14100 172.21.15.19:0/3985619028' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T11:57:15.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:15 smithi019 ceph-mon[29213]: from='mgr.14100 172.21.15.19:0/3985619028' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T11:57:15.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:15 smithi019 ceph-mon[29213]: from='mgr.14100 172.21.15.19:0/3985619028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:57:15.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:15 smithi019 ceph-mon[29213]: from='mgr.14100 172.21.15.19:0/3985619028' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T11:57:15.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:15 smithi019 ceph-mon[29213]: Manager daemon x is now available 2024-04-25T11:57:15.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:15 smithi019 ceph-mon[29213]: from='mgr.14100 172.21.15.19:0/3985619028' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T11:57:15.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:15 smithi019 ceph-mon[29213]: from='mgr.14100 172.21.15.19:0/3985619028' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T11:57:15.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:15 smithi019 ceph-mon[29213]: from='mgr.14100 172.21.15.19:0/3985619028' entity='mgr.x' 2024-04-25T11:57:15.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:15 smithi019 ceph-mon[29213]: from='mgr.14100 172.21.15.19:0/3985619028' entity='mgr.x' 2024-04-25T11:57:15.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:15 smithi019 ceph-mon[29213]: from='mgr.14100 172.21.15.19:0/3985619028' entity='mgr.x' 2024-04-25T11:57:16.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:16 smithi019 ceph-mon[29213]: mgrmap e3: x(active, since 1.00735s) 2024-04-25T11:57:17.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:17 smithi019 ceph-mon[29213]: mgrmap e4: x(active, since 2s) 2024-04-25T11:57:18.155 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T11:57:18.155 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:57:18.155 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "be62e290-02fa-11ef-bc93-c7b262605968", 2024-04-25T11:57:18.155 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T11:57:18.155 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T11:57:18.155 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T11:57:18.155 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T11:57:18.155 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T11:57:18.156 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:18.157 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T11:57:18.157 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T11:57:18.157 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T11:57:18.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T11:57:18.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T11:57:18.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T11:57:18.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:18.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T11:57:18.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T11:57:18.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T11:57:18.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T11:57:18.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T11:57:18.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T11:57:18.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T11:57:18.165 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T11:57:18.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T11:57:18.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:18.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T11:57:18.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:18.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T11:57:18.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T11:57:18.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:18.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T11:57:18.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T11:57:18.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T11:57:18.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T11:57:18.166 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T11:57:18.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T11:57:18.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T11:57:18.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:57:18.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:57:18.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:18.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T11:57:18.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:57:18.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T11:57:00.082903+0000", 2024-04-25T11:57:18.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:57:18.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:57:18.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T11:57:18.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:57:18.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.305+0000 7fa1b5e39700 1 Processor -- start 2024-04-25T11:57:18.167 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.305+0000 7fa1b5e39700 1 -- start start 2024-04-25T11:57:18.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.306+0000 7fa1b5e39700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0109090 0x7fa1b010b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:18.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.306+0000 7fa1b5e39700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b007be20 con 0x7fa1b0109090 2024-04-25T11:57:18.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.306+0000 7fa1af7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0109090 0x7fa1b010b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:18.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.306+0000 7fa1af7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0109090 0x7fa1b010b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58516/0 (socket says 172.21.15.19:58516) 2024-04-25T11:57:18.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.306+0000 7fa1af7fe700 1 -- 172.21.15.19:0/2415786537 learned_addr learned my addr 172.21.15.19:0/2415786537 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:18.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.306+0000 7fa1af7fe700 1 -- 172.21.15.19:0/2415786537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b010b9c0 con 0x7fa1b0109090 2024-04-25T11:57:18.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.306+0000 7fa1af7fe700 1 --2- 172.21.15.19:0/2415786537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0109090 0x7fa1b010b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00098a0 tx=0x7fa1a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21f3ce799356907 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:18.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.307+0000 7fa1ae7fc700 1 -- 172.21.15.19:0/2415786537 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a0004160 con 0x7fa1b0109090 2024-04-25T11:57:18.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.307+0000 7fa1ae7fc700 1 -- 172.21.15.19:0/2415786537 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fa1a0004300 con 0x7fa1b0109090 2024-04-25T11:57:18.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.307+0000 7fa1ae7fc700 1 -- 172.21.15.19:0/2415786537 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a0011b20 con 0x7fa1b0109090 2024-04-25T11:57:18.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.307+0000 7fa1b5e39700 1 -- 172.21.15.19:0/2415786537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0109090 msgr2=0x7fa1b010b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:18.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.307+0000 7fa1b5e39700 1 --2- 172.21.15.19:0/2415786537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0109090 0x7fa1b010b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00098a0 tx=0x7fa1a0009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:18.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.307+0000 7fa1b5e39700 1 -- 172.21.15.19:0/2415786537 shutdown_connections 2024-04-25T11:57:18.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.307+0000 7fa1b5e39700 1 --2- 172.21.15.19:0/2415786537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0109090 0x7fa1b010b480 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:18.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.307+0000 7fa1b5e39700 1 -- 172.21.15.19:0/2415786537 >> 172.21.15.19:0/2415786537 conn(0x7fa1b0102a40 msgr2=0x7fa1b0104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:18.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.307+0000 7fa1b5e39700 1 -- 172.21.15.19:0/2415786537 shutdown_connections 2024-04-25T11:57:18.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.307+0000 7fa1b5e39700 1 -- 172.21.15.19:0/2415786537 wait complete. 2024-04-25T11:57:18.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.308+0000 7fa1b5e39700 1 Processor -- start 2024-04-25T11:57:18.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.308+0000 7fa1b5e39700 1 -- start start 2024-04-25T11:57:18.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.308+0000 7fa1b5e39700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0109090 0x7fa1b019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:18.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.308+0000 7fa1b5e39700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b010c980 con 0x7fa1b0109090 2024-04-25T11:57:18.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.308+0000 7fa1af7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0109090 0x7fa1b019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:18.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.308+0000 7fa1af7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0109090 0x7fa1b019c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58528/0 (socket says 172.21.15.19:58528) 2024-04-25T11:57:18.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.308+0000 7fa1af7fe700 1 -- 172.21.15.19:0/666211128 learned_addr learned my addr 172.21.15.19:0/666211128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:18.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.309+0000 7fa1af7fe700 1 -- 172.21.15.19:0/666211128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b019c950 con 0x7fa1b0109090 2024-04-25T11:57:18.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.309+0000 7fa1af7fe700 1 --2- 172.21.15.19:0/666211128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0109090 0x7fa1b019c410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0006af0 tx=0x7fa1a0011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:18.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.309+0000 7fa1acff9700 1 -- 172.21.15.19:0/666211128 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a00039a0 con 0x7fa1b0109090 2024-04-25T11:57:18.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.309+0000 7fa1b5e39700 1 -- 172.21.15.19:0/666211128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b019cbe0 con 0x7fa1b0109090 2024-04-25T11:57:18.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.309+0000 7fa1acff9700 1 -- 172.21.15.19:0/666211128 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fa1a0003b40 con 0x7fa1b0109090 2024-04-25T11:57:18.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.309+0000 7fa1acff9700 1 -- 172.21.15.19:0/666211128 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a0019670 con 0x7fa1b0109090 2024-04-25T11:57:18.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.309+0000 7fa1b5e39700 1 -- 172.21.15.19:0/666211128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b019d040 con 0x7fa1b0109090 2024-04-25T11:57:18.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.310+0000 7fa1acff9700 1 -- 172.21.15.19:0/666211128 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fa1a0019950 con 0x7fa1b0109090 2024-04-25T11:57:18.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.310+0000 7fa1acff9700 1 --2- 172.21.15.19:0/666211128 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa1980395d0 0x7fa19803ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:18.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.310+0000 7fa1acff9700 1 -- 172.21.15.19:0/666211128 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa1a004dbd0 con 0x7fa1b0109090 2024-04-25T11:57:18.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.310+0000 7fa1b5e39700 1 -- 172.21.15.19:0/666211128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b00640a0 con 0x7fa1b0109090 2024-04-25T11:57:18.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.310+0000 7fa1aeffd700 1 --2- 172.21.15.19:0/666211128 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa1980395d0 0x7fa19803ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:18.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.311+0000 7fa1aeffd700 1 --2- 172.21.15.19:0/666211128 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa1980395d0 0x7fa19803ba90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4006fd0 tx=0x7fa1a4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:18.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.316+0000 7fa1acff9700 1 -- 172.21.15.19:0/666211128 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1a0011e20 con 0x7fa1b0109090 2024-04-25T11:57:18.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.611+0000 7fa1b5e39700 1 -- 172.21.15.19:0/666211128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa1b019fad0 con 0x7fa1b0109090 2024-04-25T11:57:18.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.612+0000 7fa1acff9700 1 -- 172.21.15.19:0/666211128 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa1a0003cf0 con 0x7fa1b0109090 2024-04-25T11:57:18.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.614+0000 7fa1b5e39700 1 -- 172.21.15.19:0/666211128 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa1980395d0 msgr2=0x7fa19803ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:18.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.614+0000 7fa1b5e39700 1 --2- 172.21.15.19:0/666211128 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa1980395d0 0x7fa19803ba90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4006fd0 tx=0x7fa1a4006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:18.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.614+0000 7fa1b5e39700 1 -- 172.21.15.19:0/666211128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0109090 msgr2=0x7fa1b019c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:18.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.614+0000 7fa1b5e39700 1 --2- 172.21.15.19:0/666211128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0109090 0x7fa1b019c410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0006af0 tx=0x7fa1a0011ce0 comp rx=0 tx=0).stop 2024-04-25T11:57:18.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.614+0000 7fa1b5e39700 1 -- 172.21.15.19:0/666211128 shutdown_connections 2024-04-25T11:57:18.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.614+0000 7fa1b5e39700 1 --2- 172.21.15.19:0/666211128 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa1980395d0 0x7fa19803ba90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:18.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.614+0000 7fa1b5e39700 1 --2- 172.21.15.19:0/666211128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1b0109090 0x7fa1b019c410 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:18.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.614+0000 7fa1b5e39700 1 -- 172.21.15.19:0/666211128 >> 172.21.15.19:0/666211128 conn(0x7fa1b0102a40 msgr2=0x7fa1b0103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:18.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.614+0000 7fa1b5e39700 1 -- 172.21.15.19:0/666211128 shutdown_connections 2024-04-25T11:57:18.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:17.614+0000 7fa1b5e39700 1 -- 172.21.15.19:0/666211128 wait complete. 2024-04-25T11:57:18.171 INFO:teuthology.orchestra.run.smithi019.stdout:mgr is available 2024-04-25T11:57:18.402 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:18 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/666211128' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T11:57:19.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T11:57:19.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T11:57:19.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:57:19.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T11:57:19.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T11:57:19.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T11:57:19.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T11:57:19.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T11:57:19.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T11:57:19.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T11:57:19.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T11:57:19.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-04-25T11:57:19.391 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T11:57:19.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T11:57:19.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T11:57:19.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T11:57:19.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T11:57:19.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T11:57:19.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.613+0000 7fe3d3f62700 1 Processor -- start 2024-04-25T11:57:19.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.613+0000 7fe3d3f62700 1 -- start start 2024-04-25T11:57:19.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.613+0000 7fe3d3f62700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc108650 0x7fe3cc108a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:19.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.613+0000 7fe3d3f62700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc109040 con 0x7fe3cc108650 2024-04-25T11:57:19.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.613+0000 7fe3d1cfe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc108650 0x7fe3cc108a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:19.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.613+0000 7fe3d1cfe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc108650 0x7fe3cc108a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58530/0 (socket says 172.21.15.19:58530) 2024-04-25T11:57:19.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.613+0000 7fe3d1cfe700 1 -- 172.21.15.19:0/2979968767 learned_addr learned my addr 172.21.15.19:0/2979968767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:19.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.614+0000 7fe3d1cfe700 1 -- 172.21.15.19:0/2979968767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3cc109800 con 0x7fe3cc108650 2024-04-25T11:57:19.392 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.614+0000 7fe3d1cfe700 1 --2- 172.21.15.19:0/2979968767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc108650 0x7fe3cc108a70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80098a0 tx=0x7fe3c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5caa238f59621ba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:19.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.614+0000 7fe3d0cfc700 1 -- 172.21.15.19:0/2979968767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3c8004160 con 0x7fe3cc108650 2024-04-25T11:57:19.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.614+0000 7fe3d0cfc700 1 -- 172.21.15.19:0/2979968767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fe3c8004300 con 0x7fe3cc108650 2024-04-25T11:57:19.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.614+0000 7fe3d0cfc700 1 -- 172.21.15.19:0/2979968767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3c8011b20 con 0x7fe3cc108650 2024-04-25T11:57:19.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.615+0000 7fe3d3f62700 1 -- 172.21.15.19:0/2979968767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc108650 msgr2=0x7fe3cc108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:19.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.615+0000 7fe3d3f62700 1 --2- 172.21.15.19:0/2979968767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc108650 0x7fe3cc108a70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80098a0 tx=0x7fe3c8009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:19.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.615+0000 7fe3d3f62700 1 -- 172.21.15.19:0/2979968767 shutdown_connections 2024-04-25T11:57:19.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.615+0000 7fe3d3f62700 1 --2- 172.21.15.19:0/2979968767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc108650 0x7fe3cc108a70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:19.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.615+0000 7fe3d3f62700 1 -- 172.21.15.19:0/2979968767 >> 172.21.15.19:0/2979968767 conn(0x7fe3cc103cb0 msgr2=0x7fe3cc1060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:19.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.615+0000 7fe3d3f62700 1 -- 172.21.15.19:0/2979968767 shutdown_connections 2024-04-25T11:57:19.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.615+0000 7fe3d3f62700 1 -- 172.21.15.19:0/2979968767 wait complete. 2024-04-25T11:57:19.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.615+0000 7fe3d3f62700 1 Processor -- start 2024-04-25T11:57:19.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.615+0000 7fe3d3f62700 1 -- start start 2024-04-25T11:57:19.393 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.616+0000 7fe3d3f62700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc108650 0x7fe3cc19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:19.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.616+0000 7fe3d3f62700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc10cbb0 con 0x7fe3cc108650 2024-04-25T11:57:19.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.616+0000 7fe3d1cfe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc108650 0x7fe3cc19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:19.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.616+0000 7fe3d1cfe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc108650 0x7fe3cc19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58540/0 (socket says 172.21.15.19:58540) 2024-04-25T11:57:19.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.616+0000 7fe3d1cfe700 1 -- 172.21.15.19:0/3387933820 learned_addr learned my addr 172.21.15.19:0/3387933820 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:19.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.616+0000 7fe3d1cfe700 1 -- 172.21.15.19:0/3387933820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3cc19c960 con 0x7fe3cc108650 2024-04-25T11:57:19.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.616+0000 7fe3d1cfe700 1 --2- 172.21.15.19:0/3387933820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc108650 0x7fe3cc19c420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8006af0 tx=0x7fe3c8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:19.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.616+0000 7fe3c2ffd700 1 -- 172.21.15.19:0/3387933820 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3c8003930 con 0x7fe3cc108650 2024-04-25T11:57:19.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.616+0000 7fe3c2ffd700 1 -- 172.21.15.19:0/3387933820 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fe3c8003ad0 con 0x7fe3cc108650 2024-04-25T11:57:19.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.616+0000 7fe3d3f62700 1 -- 172.21.15.19:0/3387933820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3cc19cbf0 con 0x7fe3cc108650 2024-04-25T11:57:19.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.616+0000 7fe3c2ffd700 1 -- 172.21.15.19:0/3387933820 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3c8019610 con 0x7fe3cc108650 2024-04-25T11:57:19.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.616+0000 7fe3d3f62700 1 -- 172.21.15.19:0/3387933820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3cc19d050 con 0x7fe3cc108650 2024-04-25T11:57:19.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.617+0000 7fe3c2ffd700 1 -- 172.21.15.19:0/3387933820 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fe3c801aa00 con 0x7fe3cc108650 2024-04-25T11:57:19.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.617+0000 7fe3c2ffd700 1 --2- 172.21.15.19:0/3387933820 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fe3b8039620 0x7fe3b803bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:19.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.617+0000 7fe3d3f62700 1 -- 172.21.15.19:0/3387933820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3cc0640a0 con 0x7fe3cc108650 2024-04-25T11:57:19.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.617+0000 7fe3c2ffd700 1 -- 172.21.15.19:0/3387933820 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe3c804da60 con 0x7fe3cc108650 2024-04-25T11:57:19.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.617+0000 7fe3d14fd700 1 --2- 172.21.15.19:0/3387933820 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fe3b8039620 0x7fe3b803bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:19.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.618+0000 7fe3d14fd700 1 --2- 172.21.15.19:0/3387933820 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fe3b8039620 0x7fe3b803bae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc006fd0 tx=0x7fe3bc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:19.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.622+0000 7fe3c2ffd700 1 -- 172.21.15.19:0/3387933820 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe3c8011e20 con 0x7fe3cc108650 2024-04-25T11:57:19.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.830+0000 7fe3d3f62700 1 -- 172.21.15.19:0/3387933820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe3cc19f8e0 con 0x7fe3cc108650 2024-04-25T11:57:19.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.831+0000 7fe3c2ffd700 1 -- 172.21.15.19:0/3387933820 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fe3c80172b0 con 0x7fe3cc108650 2024-04-25T11:57:19.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.833+0000 7fe3d3f62700 1 -- 172.21.15.19:0/3387933820 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fe3b8039620 msgr2=0x7fe3b803bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:19.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.833+0000 7fe3d3f62700 1 --2- 172.21.15.19:0/3387933820 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fe3b8039620 0x7fe3b803bae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc006fd0 tx=0x7fe3bc006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:19.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.833+0000 7fe3d3f62700 1 -- 172.21.15.19:0/3387933820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc108650 msgr2=0x7fe3cc19c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:19.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.833+0000 7fe3d3f62700 1 --2- 172.21.15.19:0/3387933820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc108650 0x7fe3cc19c420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8006af0 tx=0x7fe3c8011c10 comp rx=0 tx=0).stop 2024-04-25T11:57:19.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.833+0000 7fe3d3f62700 1 -- 172.21.15.19:0/3387933820 shutdown_connections 2024-04-25T11:57:19.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.833+0000 7fe3d3f62700 1 --2- 172.21.15.19:0/3387933820 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fe3b8039620 0x7fe3b803bae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:19.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.833+0000 7fe3d3f62700 1 --2- 172.21.15.19:0/3387933820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3cc108650 0x7fe3cc19c420 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:19.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.833+0000 7fe3d3f62700 1 -- 172.21.15.19:0/3387933820 >> 172.21.15.19:0/3387933820 conn(0x7fe3cc103cb0 msgr2=0x7fe3cc1046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:19.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.833+0000 7fe3d3f62700 1 -- 172.21.15.19:0/3387933820 shutdown_connections 2024-04-25T11:57:19.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:18.833+0000 7fe3d3f62700 1 -- 172.21.15.19:0/3387933820 wait complete. 2024-04-25T11:57:19.396 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling cephadm module... 2024-04-25T11:57:19.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:19 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3387933820' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T11:57:20.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:20 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1912294229' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T11:57:20.696 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:20 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: ignoring --setuser ceph since I am not root 2024-04-25T11:57:20.696 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:20 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: ignoring --setgroup ceph since I am not root 2024-04-25T11:57:20.696 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:20 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:20.574+0000 7f89c526d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:57:20.988 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:20 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:20.771+0000 7f89c526d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:57:20.988 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:20 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:20.877+0000 7f89c526d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:57:20.995 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.820+0000 7f45ab4e7700 1 Processor -- start 2024-04-25T11:57:20.995 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.820+0000 7f45ab4e7700 1 -- start start 2024-04-25T11:57:20.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.820+0000 7f45ab4e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45a41074a0 0x7f45a41078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:20.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.820+0000 7f45ab4e7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45a4107e90 con 0x7f45a41074a0 2024-04-25T11:57:20.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.820+0000 7f45a9283700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45a41074a0 0x7f45a41078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:20.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.820+0000 7f45a9283700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45a41074a0 0x7f45a41078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58546/0 (socket says 172.21.15.19:58546) 2024-04-25T11:57:20.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.820+0000 7f45a9283700 1 -- 172.21.15.19:0/597119368 learned_addr learned my addr 172.21.15.19:0/597119368 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:20.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.821+0000 7f45a9283700 1 -- 172.21.15.19:0/597119368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a41086a0 con 0x7f45a41074a0 2024-04-25T11:57:20.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.821+0000 7f45a9283700 1 --2- 172.21.15.19:0/597119368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45a41074a0 0x7f45a41078c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f45a00098a0 tx=0x7f45a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e973da2bc452bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:20.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.821+0000 7f459bfff700 1 -- 172.21.15.19:0/597119368 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45a0004160 con 0x7f45a41074a0 2024-04-25T11:57:20.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.821+0000 7f459bfff700 1 -- 172.21.15.19:0/597119368 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f45a0004300 con 0x7f45a41074a0 2024-04-25T11:57:20.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.821+0000 7f459bfff700 1 -- 172.21.15.19:0/597119368 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45a0011b20 con 0x7f45a41074a0 2024-04-25T11:57:20.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.822+0000 7f45ab4e7700 1 -- 172.21.15.19:0/597119368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45a41074a0 msgr2=0x7f45a41078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:20.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.822+0000 7f45ab4e7700 1 --2- 172.21.15.19:0/597119368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45a41074a0 0x7f45a41078c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f45a00098a0 tx=0x7f45a0009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:20.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.822+0000 7f45ab4e7700 1 -- 172.21.15.19:0/597119368 shutdown_connections 2024-04-25T11:57:20.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.822+0000 7f45ab4e7700 1 --2- 172.21.15.19:0/597119368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45a41074a0 0x7f45a41078c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:20.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.822+0000 7f45ab4e7700 1 -- 172.21.15.19:0/597119368 >> 172.21.15.19:0/597119368 conn(0x7f45a4102a40 msgr2=0x7f45a4104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:20.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.822+0000 7f45ab4e7700 1 -- 172.21.15.19:0/597119368 shutdown_connections 2024-04-25T11:57:20.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.822+0000 7f45ab4e7700 1 -- 172.21.15.19:0/597119368 wait complete. 2024-04-25T11:57:20.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.822+0000 7f45ab4e7700 1 Processor -- start 2024-04-25T11:57:20.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.822+0000 7f45ab4e7700 1 -- start start 2024-04-25T11:57:20.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.823+0000 7f45ab4e7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45a41074a0 0x7f45a419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:20.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.823+0000 7f45ab4e7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f45a410ba50 con 0x7f45a41074a0 2024-04-25T11:57:20.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.823+0000 7f45a9283700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45a41074a0 0x7f45a419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:20.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.823+0000 7f45a9283700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45a41074a0 0x7f45a419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58560/0 (socket says 172.21.15.19:58560) 2024-04-25T11:57:20.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.823+0000 7f45a9283700 1 -- 172.21.15.19:0/1912294229 learned_addr learned my addr 172.21.15.19:0/1912294229 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:20.997 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.823+0000 7f45a9283700 1 -- 172.21.15.19:0/1912294229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a419b890 con 0x7f45a41074a0 2024-04-25T11:57:20.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.823+0000 7f45a9283700 1 --2- 172.21.15.19:0/1912294229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45a41074a0 0x7f45a419b350 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f45a00117e0 tx=0x7f45a0011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:20.998 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.823+0000 7f459a7fc700 1 -- 172.21.15.19:0/1912294229 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45a0003b30 con 0x7f45a41074a0 2024-04-25T11:57:20.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.823+0000 7f459a7fc700 1 -- 172.21.15.19:0/1912294229 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f45a0003cd0 con 0x7f45a41074a0 2024-04-25T11:57:20.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.823+0000 7f45ab4e7700 1 -- 172.21.15.19:0/1912294229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a419bb20 con 0x7f45a41074a0 2024-04-25T11:57:20.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.823+0000 7f459a7fc700 1 -- 172.21.15.19:0/1912294229 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45a001a830 con 0x7f45a41074a0 2024-04-25T11:57:20.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.823+0000 7f45ab4e7700 1 -- 172.21.15.19:0/1912294229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a419bf80 con 0x7f45a41074a0 2024-04-25T11:57:20.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.824+0000 7f459a7fc700 1 -- 172.21.15.19:0/1912294229 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f45a0024440 con 0x7f45a41074a0 2024-04-25T11:57:20.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.824+0000 7f45ab4e7700 1 -- 172.21.15.19:0/1912294229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a40640a0 con 0x7f45a41074a0 2024-04-25T11:57:20.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.824+0000 7f459a7fc700 1 --2- 172.21.15.19:0/1912294229 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7f4590039620 0x7f459003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:20.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.824+0000 7f459a7fc700 1 -- 172.21.15.19:0/1912294229 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f45a004d990 con 0x7f45a41074a0 2024-04-25T11:57:20.999 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.825+0000 7f45a8a82700 1 --2- 172.21.15.19:0/1912294229 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7f4590039620 0x7f459003bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:21.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.825+0000 7f45a8a82700 1 --2- 172.21.15.19:0/1912294229 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7f4590039620 0x7f459003bae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4594006fd0 tx=0x7f4594006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:21.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:19.830+0000 7f459a7fc700 1 -- 172.21.15.19:0/1912294229 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45a000e260 con 0x7f45a41074a0 2024-04-25T11:57:21.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:20.093+0000 7f45ab4e7700 1 -- 172.21.15.19:0/1912294229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f45a419e950 con 0x7f45a41074a0 2024-04-25T11:57:21.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:20.386+0000 7f459a7fc700 1 -- 172.21.15.19:0/1912294229 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f45a0024730 con 0x7f45a41074a0 2024-04-25T11:57:21.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:20.388+0000 7f459a7fc700 1 -- 172.21.15.19:0/1912294229 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f45a002d3f0 con 0x7f45a41074a0 2024-04-25T11:57:21.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:20.389+0000 7f45ab4e7700 1 -- 172.21.15.19:0/1912294229 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7f4590039620 msgr2=0x7f459003bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:21.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:20.389+0000 7f45ab4e7700 1 --2- 172.21.15.19:0/1912294229 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7f4590039620 0x7f459003bae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4594006fd0 tx=0x7f4594006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:21.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:20.389+0000 7f45ab4e7700 1 -- 172.21.15.19:0/1912294229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45a41074a0 msgr2=0x7f45a419b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:21.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:20.389+0000 7f45ab4e7700 1 --2- 172.21.15.19:0/1912294229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45a41074a0 0x7f45a419b350 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f45a00117e0 tx=0x7f45a0011ce0 comp rx=0 tx=0).stop 2024-04-25T11:57:21.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:20.390+0000 7f45ab4e7700 1 -- 172.21.15.19:0/1912294229 shutdown_connections 2024-04-25T11:57:21.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:20.390+0000 7f45ab4e7700 1 --2- 172.21.15.19:0/1912294229 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7f4590039620 0x7f459003bae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:21.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:20.390+0000 7f45ab4e7700 1 --2- 172.21.15.19:0/1912294229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f45a41074a0 0x7f45a419b350 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:21.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:20.390+0000 7f45ab4e7700 1 -- 172.21.15.19:0/1912294229 >> 172.21.15.19:0/1912294229 conn(0x7f45a4102a40 msgr2=0x7f45a4103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:21.000 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:20.390+0000 7f45ab4e7700 1 -- 172.21.15.19:0/1912294229 shutdown_connections 2024-04-25T11:57:21.001 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:20.390+0000 7f45ab4e7700 1 -- 172.21.15.19:0/1912294229 wait complete. 2024-04-25T11:57:21.246 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:21 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:21.211+0000 7f89c526d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:57:21.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:21 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1912294229' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T11:57:21.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:21 smithi019 ceph-mon[29213]: mgrmap e5: x(active, since 6s) 2024-04-25T11:57:21.542 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:21 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:21.306+0000 7f89c526d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:57:22.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:57:22.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T11:57:22.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T11:57:22.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T11:57:22.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T11:57:22.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:57:22.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.424+0000 7fc2bca1f700 1 Processor -- start 2024-04-25T11:57:22.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.424+0000 7fc2bca1f700 1 -- start start 2024-04-25T11:57:22.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.425+0000 7fc2bca1f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2b8106e60 0x7fc2b8107280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:22.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.425+0000 7fc2bca1f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b807be40 con 0x7fc2b8106e60 2024-04-25T11:57:22.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.425+0000 7fc2b659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2b8106e60 0x7fc2b8107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:22.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.425+0000 7fc2b659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2b8106e60 0x7fc2b8107280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58596/0 (socket says 172.21.15.19:58596) 2024-04-25T11:57:22.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.425+0000 7fc2b659c700 1 -- 172.21.15.19:0/1607749006 learned_addr learned my addr 172.21.15.19:0/1607749006 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:22.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.425+0000 7fc2b659c700 1 -- 172.21.15.19:0/1607749006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b81077c0 con 0x7fc2b8106e60 2024-04-25T11:57:22.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.425+0000 7fc2b659c700 1 --2- 172.21.15.19:0/1607749006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2b8106e60 0x7fc2b8107280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80098a0 tx=0x7fc2a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fddb1b5a82259f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:22.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.426+0000 7fc2b559a700 1 -- 172.21.15.19:0/1607749006 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2a8004160 con 0x7fc2b8106e60 2024-04-25T11:57:22.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.426+0000 7fc2b559a700 1 -- 172.21.15.19:0/1607749006 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fc2a8004300 con 0x7fc2b8106e60 2024-04-25T11:57:22.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.426+0000 7fc2b559a700 1 -- 172.21.15.19:0/1607749006 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2a8011b20 con 0x7fc2b8106e60 2024-04-25T11:57:22.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.426+0000 7fc2bca1f700 1 -- 172.21.15.19:0/1607749006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2b8106e60 msgr2=0x7fc2b8107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:22.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.426+0000 7fc2bca1f700 1 --2- 172.21.15.19:0/1607749006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2b8106e60 0x7fc2b8107280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80098a0 tx=0x7fc2a8009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.426+0000 7fc2bca1f700 1 -- 172.21.15.19:0/1607749006 shutdown_connections 2024-04-25T11:57:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.426+0000 7fc2bca1f700 1 --2- 172.21.15.19:0/1607749006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2b8106e60 0x7fc2b8107280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.426+0000 7fc2bca1f700 1 -- 172.21.15.19:0/1607749006 >> 172.21.15.19:0/1607749006 conn(0x7fc2b8102a40 msgr2=0x7fc2b8104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.426+0000 7fc2bca1f700 1 -- 172.21.15.19:0/1607749006 shutdown_connections 2024-04-25T11:57:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.426+0000 7fc2bca1f700 1 -- 172.21.15.19:0/1607749006 wait complete. 2024-04-25T11:57:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.427+0000 7fc2bca1f700 1 Processor -- start 2024-04-25T11:57:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.427+0000 7fc2bca1f700 1 -- start start 2024-04-25T11:57:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.427+0000 7fc2bca1f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2b8106e60 0x7fc2b819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.427+0000 7fc2bca1f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b8108850 con 0x7fc2b8106e60 2024-04-25T11:57:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.427+0000 7fc2b659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2b8106e60 0x7fc2b819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.428+0000 7fc2b659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2b8106e60 0x7fc2b819e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58610/0 (socket says 172.21.15.19:58610) 2024-04-25T11:57:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.428+0000 7fc2b659c700 1 -- 172.21.15.19:0/903142389 learned_addr learned my addr 172.21.15.19:0/903142389 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.428+0000 7fc2b659c700 1 -- 172.21.15.19:0/903142389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b819eb50 con 0x7fc2b8106e60 2024-04-25T11:57:22.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.428+0000 7fc2b659c700 1 --2- 172.21.15.19:0/903142389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2b8106e60 0x7fc2b819e610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80117e0 tx=0x7fc2a8011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.428+0000 7fc2af7fe700 1 -- 172.21.15.19:0/903142389 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2a8003b30 con 0x7fc2b8106e60 2024-04-25T11:57:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.428+0000 7fc2af7fe700 1 -- 172.21.15.19:0/903142389 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fc2a8003cd0 con 0x7fc2b8106e60 2024-04-25T11:57:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.428+0000 7fc2bca1f700 1 -- 172.21.15.19:0/903142389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b819ede0 con 0x7fc2b8106e60 2024-04-25T11:57:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.428+0000 7fc2af7fe700 1 -- 172.21.15.19:0/903142389 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2a801a7c0 con 0x7fc2b8106e60 2024-04-25T11:57:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.428+0000 7fc2bca1f700 1 -- 172.21.15.19:0/903142389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b819f240 con 0x7fc2b8106e60 2024-04-25T11:57:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.429+0000 7fc2af7fe700 1 -- 172.21.15.19:0/903142389 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fc2a801aa70 con 0x7fc2b8106e60 2024-04-25T11:57:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.429+0000 7fc2af7fe700 1 --2- 172.21.15.19:0/903142389 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fc2a4039670 0x7fc2a403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.429+0000 7fc2af7fe700 1 -- 172.21.15.19:0/903142389 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc2a804e6b0 con 0x7fc2b8106e60 2024-04-25T11:57:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.429+0000 7fc2b5d9b700 1 -- 172.21.15.19:0/903142389 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fc2a4039670 msgr2=0x7fc2a403bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/556620573 2024-04-25T11:57:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.429+0000 7fc2b5d9b700 1 --2- 172.21.15.19:0/903142389 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fc2a4039670 0x7fc2a403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:57:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.429+0000 7fc2bca1f700 1 -- 172.21.15.19:0/903142389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b80640a0 con 0x7fc2b8106e60 2024-04-25T11:57:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.434+0000 7fc2af7fe700 1 -- 172.21.15.19:0/903142389 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2a80042c0 con 0x7fc2b8106e60 2024-04-25T11:57:22.331 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.630+0000 7fc2b5d9b700 1 -- 172.21.15.19:0/903142389 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fc2a4039670 msgr2=0x7fc2a403bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/556620573 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.630+0000 7fc2b5d9b700 1 --2- 172.21.15.19:0/903142389 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fc2a4039670 0x7fc2a403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.718+0000 7fc2bca1f700 1 -- 172.21.15.19:0/903142389 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fc2b802e3d0 con 0x7fc2b8106e60 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.719+0000 7fc2af7fe700 1 -- 172.21.15.19:0/903142389 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fc2a800fdd0 con 0x7fc2b8106e60 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.723+0000 7fc2bca1f700 1 -- 172.21.15.19:0/903142389 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fc2a4039670 msgr2=0x7fc2a403bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.723+0000 7fc2bca1f700 1 --2- 172.21.15.19:0/903142389 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fc2a4039670 0x7fc2a403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.723+0000 7fc2bca1f700 1 -- 172.21.15.19:0/903142389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2b8106e60 msgr2=0x7fc2b819e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.723+0000 7fc2bca1f700 1 --2- 172.21.15.19:0/903142389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2b8106e60 0x7fc2b819e610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80117e0 tx=0x7fc2a8011ce0 comp rx=0 tx=0).stop 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.723+0000 7fc2bca1f700 1 -- 172.21.15.19:0/903142389 shutdown_connections 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.723+0000 7fc2bca1f700 1 --2- 172.21.15.19:0/903142389 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fc2a4039670 0x7fc2a403bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.723+0000 7fc2bca1f700 1 --2- 172.21.15.19:0/903142389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2b8106e60 0x7fc2b819e610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.723+0000 7fc2bca1f700 1 -- 172.21.15.19:0/903142389 >> 172.21.15.19:0/903142389 conn(0x7fc2b8102a40 msgr2=0x7fc2b81952d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.724+0000 7fc2bca1f700 1 -- 172.21.15.19:0/903142389 shutdown_connections 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:21.724+0000 7fc2bca1f700 1 -- 172.21.15.19:0/903142389 wait complete. 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-04-25T11:57:22.332 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 5... 2024-04-25T11:57:22.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:22 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/903142389' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T11:57:22.590 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:22 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:22.564+0000 7f89c526d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:57:22.918 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:22 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:22.748+0000 7f89c526d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:57:22.918 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:22 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:22.926+0000 7f89c526d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:57:23.526 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:23 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:23.229+0000 7f89c526d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:57:23.527 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:23 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:23.325+0000 7f89c526d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:57:23.527 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:23 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:23.534+0000 7f89c526d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:57:23.791 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:23 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:23.752+0000 7f89c526d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:57:24.542 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:24 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:24.375+0000 7f89c526d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:57:25.113 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:24 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:24.812+0000 7f89c526d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:57:25.113 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:24 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:24.912+0000 7f89c526d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:57:25.113 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:25 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:25.121+0000 7f89c526d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:57:26.005 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:25 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:25.665+0000 7f89c526d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:57:26.005 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:25 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:25.760+0000 7f89c526d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:57:26.292 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:26 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:26.013+0000 7f89c526d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:57:26.292 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:26 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:26.151+0000 7f89c526d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:57:26.292 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:26 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:26.250+0000 7f89c526d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:57:26.701 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:26 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:26.369+0000 7f89c526d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:57:26.701 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:26 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:26.708+0000 7f89c526d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:57:27.604 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:27.336+0000 7f89c526d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:57:27.605 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:27.559+0000 7f89c526d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:57:28.042 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: Active manager daemon x restarted 2024-04-25T11:57:28.042 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: Activating manager daemon x 2024-04-25T11:57:28.042 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T11:57:28.042 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: mgrmap e6: x(active, starting, since 0.0031449s) 2024-04-25T11:57:28.042 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:57:28.042 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T11:57:28.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T11:57:28.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T11:57:28.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T11:57:28.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: Manager daemon x is now available 2024-04-25T11:57:28.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' 2024-04-25T11:57:28.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' 2024-04-25T11:57:28.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:57:28.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:57:28.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:27 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T11:57:28.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:28 smithi019 ceph-mon[29213]: Found migration_current of "None". Setting to last migration. 2024-04-25T11:57:28.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:28 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T11:57:28.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:28 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' 2024-04-25T11:57:28.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:28 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' 2024-04-25T11:57:28.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:28 smithi019 ceph-mon[29213]: mgrmap e7: x(active, since 1.00544s) 2024-04-25T11:57:29.168 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:57:29.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T11:57:29.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T11:57:29.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:57:29.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.794+0000 7fa0b6696700 1 Processor -- start 2024-04-25T11:57:29.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.794+0000 7fa0b6696700 1 -- start start 2024-04-25T11:57:29.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.794+0000 7fa0b6696700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b01085b0 0x7fa0b01089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:29.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.794+0000 7fa0b6696700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0108fa0 con 0x7fa0b01085b0 2024-04-25T11:57:29.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.794+0000 7fa0affff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b01085b0 0x7fa0b01089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:29.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.794+0000 7fa0affff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b01085b0 0x7fa0b01089d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52720/0 (socket says 172.21.15.19:52720) 2024-04-25T11:57:29.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.794+0000 7fa0affff700 1 -- 172.21.15.19:0/2111163037 learned_addr learned my addr 172.21.15.19:0/2111163037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:29.169 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.795+0000 7fa0affff700 1 -- 172.21.15.19:0/2111163037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b0109760 con 0x7fa0b01085b0 2024-04-25T11:57:29.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.795+0000 7fa0affff700 1 --2- 172.21.15.19:0/2111163037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b01085b0 0x7fa0b01089d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00098a0 tx=0x7fa0a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e19305a173ab8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:29.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.795+0000 7fa0aeffd700 1 -- 172.21.15.19:0/2111163037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0a0004160 con 0x7fa0b01085b0 2024-04-25T11:57:29.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.795+0000 7fa0aeffd700 1 -- 172.21.15.19:0/2111163037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fa0a0004300 con 0x7fa0b01085b0 2024-04-25T11:57:29.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.795+0000 7fa0aeffd700 1 -- 172.21.15.19:0/2111163037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0a0011b20 con 0x7fa0b01085b0 2024-04-25T11:57:29.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.796+0000 7fa0b6696700 1 -- 172.21.15.19:0/2111163037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b01085b0 msgr2=0x7fa0b01089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:29.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.796+0000 7fa0b6696700 1 --2- 172.21.15.19:0/2111163037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b01085b0 0x7fa0b01089d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00098a0 tx=0x7fa0a0009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:29.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.796+0000 7fa0b6696700 1 -- 172.21.15.19:0/2111163037 shutdown_connections 2024-04-25T11:57:29.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.796+0000 7fa0b6696700 1 --2- 172.21.15.19:0/2111163037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b01085b0 0x7fa0b01089d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:29.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.796+0000 7fa0b6696700 1 -- 172.21.15.19:0/2111163037 >> 172.21.15.19:0/2111163037 conn(0x7fa0b0103bb0 msgr2=0x7fa0b0105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:29.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.796+0000 7fa0b6696700 1 -- 172.21.15.19:0/2111163037 shutdown_connections 2024-04-25T11:57:29.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.796+0000 7fa0b6696700 1 -- 172.21.15.19:0/2111163037 wait complete. 2024-04-25T11:57:29.170 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.797+0000 7fa0b6696700 1 Processor -- start 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.797+0000 7fa0b6696700 1 -- start start 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.797+0000 7fa0b6696700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b01085b0 0x7fa0b019c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.797+0000 7fa0b6696700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b010cb10 con 0x7fa0b01085b0 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.797+0000 7fa0affff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b01085b0 0x7fa0b019c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.797+0000 7fa0affff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b01085b0 0x7fa0b019c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52724/0 (socket says 172.21.15.19:52724) 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.797+0000 7fa0affff700 1 -- 172.21.15.19:0/3246595613 learned_addr learned my addr 172.21.15.19:0/3246595613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.798+0000 7fa0affff700 1 -- 172.21.15.19:0/3246595613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b019c910 con 0x7fa0b01085b0 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.798+0000 7fa0affff700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b01085b0 0x7fa0b019c3d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00117e0 tx=0x7fa0a0011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.798+0000 7fa0ad7fa700 1 -- 172.21.15.19:0/3246595613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0a0003b30 con 0x7fa0b01085b0 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.798+0000 7fa0ad7fa700 1 -- 172.21.15.19:0/3246595613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fa0a0003cd0 con 0x7fa0b01085b0 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.798+0000 7fa0b6696700 1 -- 172.21.15.19:0/3246595613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b019cba0 con 0x7fa0b01085b0 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.798+0000 7fa0ad7fa700 1 -- 172.21.15.19:0/3246595613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0a001a830 con 0x7fa0b01085b0 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.798+0000 7fa0b6696700 1 -- 172.21.15.19:0/3246595613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b019d000 con 0x7fa0b01085b0 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.799+0000 7fa0ad7fa700 1 -- 172.21.15.19:0/3246595613 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fa0a001a9d0 con 0x7fa0b01085b0 2024-04-25T11:57:29.171 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.799+0000 7fa0ad7fa700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa098039670 0x7fa09803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.799+0000 7fa0ad7fa700 1 -- 172.21.15.19:0/3246595613 --> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa09803c240 con 0x7fa098039670 2024-04-25T11:57:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.799+0000 7fa0ad7fa700 1 -- 172.21.15.19:0/3246595613 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa0a004d9e0 con 0x7fa0b01085b0 2024-04-25T11:57:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.799+0000 7fa0af7fe700 1 -- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa098039670 msgr2=0x7fa09803bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/556620573 2024-04-25T11:57:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:22.799+0000 7fa0af7fe700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa098039670 0x7fa09803bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:57:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:23.000+0000 7fa0af7fe700 1 -- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa098039670 msgr2=0x7fa09803bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/556620573 2024-04-25T11:57:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:23.000+0000 7fa0af7fe700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa098039670 0x7fa09803bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:57:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:23.401+0000 7fa0af7fe700 1 -- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa098039670 msgr2=0x7fa09803bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/556620573 2024-04-25T11:57:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:23.401+0000 7fa0af7fe700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa098039670 0x7fa09803bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:57:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:24.202+0000 7fa0af7fe700 1 -- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa098039670 msgr2=0x7fa09803bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/556620573 2024-04-25T11:57:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:24.202+0000 7fa0af7fe700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa098039670 0x7fa09803bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:57:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:25.803+0000 7fa0af7fe700 1 -- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa098039670 msgr2=0x7fa09803bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/556620573 2024-04-25T11:57:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:25.803+0000 7fa0af7fe700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa098039670 0x7fa09803bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T11:57:29.172 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:27.563+0000 7fa0ad7fa700 1 -- 172.21.15.19:0/3246595613 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fa0a002d910 con 0x7fa0b01085b0 2024-04-25T11:57:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:27.563+0000 7fa0ad7fa700 1 -- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa098039670 msgr2=0x7fa09803bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:57:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:27.563+0000 7fa0ad7fa700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa098039670 0x7fa09803bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.564+0000 7fa0ad7fa700 1 -- 172.21.15.19:0/3246595613 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fa0a0029cf0 con 0x7fa0b01085b0 2024-04-25T11:57:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.564+0000 7fa0ad7fa700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fa09803cf60 0x7fa09803f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.565+0000 7fa0ad7fa700 1 -- 172.21.15.19:0/3246595613 --> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa0a002ee50 con 0x7fa09803cf60 2024-04-25T11:57:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.565+0000 7fa0af7fe700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fa09803cf60 0x7fa09803f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.565+0000 7fa0af7fe700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fa09803cf60 0x7fa09803f350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4003a10 tx=0x7fa0a4009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.566+0000 7fa0ad7fa700 1 -- 172.21.15.19:0/3246595613 <== mgr.14120 v2:172.21.15.19:6800/963006875 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fa0a002ee50 con 0x7fa09803cf60 2024-04-25T11:57:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.577+0000 7fa0b6696700 1 -- 172.21.15.19:0/3246595613 --> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa0b00640a0 con 0x7fa09803cf60 2024-04-25T11:57:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.578+0000 7fa0ad7fa700 1 -- 172.21.15.19:0/3246595613 <== mgr.14120 v2:172.21.15.19:6800/963006875 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fa0b00640a0 con 0x7fa09803cf60 2024-04-25T11:57:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.578+0000 7fa0b6696700 1 -- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fa09803cf60 msgr2=0x7fa09803f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.578+0000 7fa0b6696700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fa09803cf60 0x7fa09803f350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4003a10 tx=0x7fa0a4009290 comp rx=0 tx=0).stop 2024-04-25T11:57:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.578+0000 7fa0b6696700 1 -- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b01085b0 msgr2=0x7fa0b019c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:29.173 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.578+0000 7fa0b6696700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b01085b0 0x7fa0b019c3d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00117e0 tx=0x7fa0a0011ce0 comp rx=0 tx=0).stop 2024-04-25T11:57:29.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.578+0000 7fa0b6696700 1 -- 172.21.15.19:0/3246595613 shutdown_connections 2024-04-25T11:57:29.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.578+0000 7fa0b6696700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fa09803cf60 0x7fa09803f350 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:29.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.578+0000 7fa0b6696700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b01085b0 0x7fa0b019c3d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:29.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.578+0000 7fa0b6696700 1 --2- 172.21.15.19:0/3246595613 >> [v2:172.21.15.19:6800/556620573,v1:172.21.15.19:6801/556620573] conn(0x7fa098039670 0x7fa09803bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:29.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.578+0000 7fa0b6696700 1 -- 172.21.15.19:0/3246595613 >> 172.21.15.19:0/3246595613 conn(0x7fa0b0103bb0 msgr2=0x7fa0b01045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:29.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.578+0000 7fa0b6696700 1 -- 172.21.15.19:0/3246595613 shutdown_connections 2024-04-25T11:57:29.174 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:28.578+0000 7fa0b6696700 1 -- 172.21.15.19:0/3246595613 wait complete. 2024-04-25T11:57:29.174 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 5 is available 2024-04-25T11:57:29.174 INFO:teuthology.orchestra.run.smithi019.stdout:Setting orchestrator backend to cephadm... 2024-04-25T11:57:29.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:29 smithi019 ceph-mon[29213]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T11:57:29.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:29 smithi019 ceph-mon[29213]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T11:57:30.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.571+0000 7f3bc6b99700 1 Processor -- start 2024-04-25T11:57:30.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.571+0000 7f3bc6b99700 1 -- start start 2024-04-25T11:57:30.374 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.571+0000 7f3bc6b99700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bc0109060 0x7f3bc010b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.571+0000 7f3bc6b99700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc007be40 con 0x7f3bc0109060 2024-04-25T11:57:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.572+0000 7f3bc4935700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bc0109060 0x7f3bc010b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.572+0000 7f3bc4935700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bc0109060 0x7f3bc010b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52802/0 (socket says 172.21.15.19:52802) 2024-04-25T11:57:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.572+0000 7f3bc4935700 1 -- 172.21.15.19:0/3088001939 learned_addr learned my addr 172.21.15.19:0/3088001939 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.572+0000 7f3bc4935700 1 -- 172.21.15.19:0/3088001939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc010b990 con 0x7f3bc0109060 2024-04-25T11:57:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.572+0000 7f3bc4935700 1 --2- 172.21.15.19:0/3088001939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bc0109060 0x7f3bc010b450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40098a0 tx=0x7f3bb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43476d75ffaa51b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.572+0000 7f3bbf7fe700 1 -- 172.21.15.19:0/3088001939 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bb4004160 con 0x7f3bc0109060 2024-04-25T11:57:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.572+0000 7f3bbf7fe700 1 -- 172.21.15.19:0/3088001939 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f3bb4004300 con 0x7f3bc0109060 2024-04-25T11:57:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.572+0000 7f3bbf7fe700 1 -- 172.21.15.19:0/3088001939 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bb4011b20 con 0x7f3bc0109060 2024-04-25T11:57:30.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.573+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3088001939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bc0109060 msgr2=0x7f3bc010b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.573+0000 7f3bc6b99700 1 --2- 172.21.15.19:0/3088001939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bc0109060 0x7f3bc010b450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40098a0 tx=0x7f3bb4009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.573+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3088001939 shutdown_connections 2024-04-25T11:57:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.573+0000 7f3bc6b99700 1 --2- 172.21.15.19:0/3088001939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bc0109060 0x7f3bc010b450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.573+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3088001939 >> 172.21.15.19:0/3088001939 conn(0x7f3bc0102a10 msgr2=0x7f3bc0104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.573+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3088001939 shutdown_connections 2024-04-25T11:57:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.573+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3088001939 wait complete. 2024-04-25T11:57:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.574+0000 7f3bc6b99700 1 Processor -- start 2024-04-25T11:57:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.574+0000 7f3bc6b99700 1 -- start start 2024-04-25T11:57:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.574+0000 7f3bc6b99700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bc0109060 0x7f3bc019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.574+0000 7f3bc6b99700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc010c960 con 0x7f3bc0109060 2024-04-25T11:57:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.574+0000 7f3bc4935700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bc0109060 0x7f3bc019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:30.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.574+0000 7f3bc4935700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bc0109060 0x7f3bc019c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52818/0 (socket says 172.21.15.19:52818) 2024-04-25T11:57:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.574+0000 7f3bc4935700 1 -- 172.21.15.19:0/3699904966 learned_addr learned my addr 172.21.15.19:0/3699904966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.574+0000 7f3bc4935700 1 -- 172.21.15.19:0/3699904966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc019c930 con 0x7f3bc0109060 2024-04-25T11:57:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.575+0000 7f3bc4935700 1 --2- 172.21.15.19:0/3699904966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bc0109060 0x7f3bc019c3f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40117e0 tx=0x7f3bb4011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.575+0000 7f3bbdffb700 1 -- 172.21.15.19:0/3699904966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bb4003b30 con 0x7f3bc0109060 2024-04-25T11:57:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.575+0000 7f3bbdffb700 1 -- 172.21.15.19:0/3699904966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f3bb4003cd0 con 0x7f3bc0109060 2024-04-25T11:57:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.575+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3699904966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc019cbc0 con 0x7f3bc0109060 2024-04-25T11:57:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.575+0000 7f3bbdffb700 1 -- 172.21.15.19:0/3699904966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bb401a7c0 con 0x7f3bc0109060 2024-04-25T11:57:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.575+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3699904966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc019d020 con 0x7f3bc0109060 2024-04-25T11:57:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.576+0000 7f3bbdffb700 1 -- 172.21.15.19:0/3699904966 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f3bb401a960 con 0x7f3bc0109060 2024-04-25T11:57:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.576+0000 7f3bbdffb700 1 --2- 172.21.15.19:0/3699904966 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f3bb00395a0 0x7f3bb003ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.576+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3699904966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc00640a0 con 0x7f3bc0109060 2024-04-25T11:57:30.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.576+0000 7f3bbdffb700 1 -- 172.21.15.19:0/3699904966 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3bb404d7f0 con 0x7f3bc0109060 2024-04-25T11:57:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.576+0000 7f3bbffff700 1 --2- 172.21.15.19:0/3699904966 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f3bb00395a0 0x7f3bb003ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.577+0000 7f3bbffff700 1 --2- 172.21.15.19:0/3699904966 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f3bb00395a0 0x7f3bb003ba60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3bac006fd0 tx=0x7f3bac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.581+0000 7f3bbdffb700 1 -- 172.21.15.19:0/3699904966 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3bb4020240 con 0x7f3bc0109060 2024-04-25T11:57:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.623+0000 7f3bbdffb700 1 -- 172.21.15.19:0/3699904966 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3bb402d910 con 0x7f3bc0109060 2024-04-25T11:57:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.808+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3699904966 --> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f3bc019f970 con 0x7f3bb00395a0 2024-04-25T11:57:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.813+0000 7f3bbdffb700 1 -- 172.21.15.19:0/3699904966 <== mgr.14120 v2:172.21.15.19:6800/963006875 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3bc019f970 con 0x7f3bb00395a0 2024-04-25T11:57:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.815+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3699904966 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f3bb00395a0 msgr2=0x7f3bb003ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.815+0000 7f3bc6b99700 1 --2- 172.21.15.19:0/3699904966 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f3bb00395a0 0x7f3bb003ba60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3bac006fd0 tx=0x7f3bac006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.815+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3699904966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bc0109060 msgr2=0x7f3bc019c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.815+0000 7f3bc6b99700 1 --2- 172.21.15.19:0/3699904966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bc0109060 0x7f3bc019c3f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40117e0 tx=0x7f3bb4011ce0 comp rx=0 tx=0).stop 2024-04-25T11:57:30.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.815+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3699904966 shutdown_connections 2024-04-25T11:57:30.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.815+0000 7f3bc6b99700 1 --2- 172.21.15.19:0/3699904966 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f3bb00395a0 0x7f3bb003ba60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:30.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.815+0000 7f3bc6b99700 1 --2- 172.21.15.19:0/3699904966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bc0109060 0x7f3bc019c3f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:30.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.815+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3699904966 >> 172.21.15.19:0/3699904966 conn(0x7f3bc0102a10 msgr2=0x7f3bc01034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:30.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.815+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3699904966 shutdown_connections 2024-04-25T11:57:30.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:29.815+0000 7f3bc6b99700 1 -- 172.21.15.19:0/3699904966 wait complete. 2024-04-25T11:57:30.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:30 smithi019 ceph-mon[29213]: mgrmap e8: x(active, since 2s) 2024-04-25T11:57:30.759 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:30 smithi019 ceph-mon[29213]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:57:30.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:30 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' 2024-04-25T11:57:30.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:30 smithi019 ceph-mon[29213]: [25/Apr/2024:11:57:29] ENGINE Bus STARTING 2024-04-25T11:57:30.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:30 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:57:30.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:30 smithi019 ceph-mon[29213]: [25/Apr/2024:11:57:29] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T11:57:30.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:30 smithi019 ceph-mon[29213]: [25/Apr/2024:11:57:30] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T11:57:30.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:30 smithi019 ceph-mon[29213]: [25/Apr/2024:11:57:30] ENGINE Bus STARTED 2024-04-25T11:57:30.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:30 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:57:31.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:31 smithi019 ceph-mon[29213]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:57:31.679 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T11:57:31.679 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.875+0000 7fa4aa180700 1 Processor -- start 2024-04-25T11:57:31.679 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.875+0000 7fa4aa180700 1 -- start start 2024-04-25T11:57:31.679 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.875+0000 7fa4aa180700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a4108640 0x7fa4a4108a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:31.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.875+0000 7fa4aa180700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a4109030 con 0x7fa4a4108640 2024-04-25T11:57:31.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.875+0000 7fa4a37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a4108640 0x7fa4a4108a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:31.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.875+0000 7fa4a37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a4108640 0x7fa4a4108a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52830/0 (socket says 172.21.15.19:52830) 2024-04-25T11:57:31.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.875+0000 7fa4a37fe700 1 -- 172.21.15.19:0/2545132049 learned_addr learned my addr 172.21.15.19:0/2545132049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:31.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.875+0000 7fa4a37fe700 1 -- 172.21.15.19:0/2545132049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a41097f0 con 0x7fa4a4108640 2024-04-25T11:57:31.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.876+0000 7fa4a37fe700 1 --2- 172.21.15.19:0/2545132049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a4108640 0x7fa4a4108a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa4940098a0 tx=0x7fa494009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1d5c6db0cbd0f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:31.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.876+0000 7fa4a27fc700 1 -- 172.21.15.19:0/2545132049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa494004160 con 0x7fa4a4108640 2024-04-25T11:57:31.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.876+0000 7fa4a27fc700 1 -- 172.21.15.19:0/2545132049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fa494004300 con 0x7fa4a4108640 2024-04-25T11:57:31.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.876+0000 7fa4a27fc700 1 -- 172.21.15.19:0/2545132049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa494011b20 con 0x7fa4a4108640 2024-04-25T11:57:31.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.876+0000 7fa4aa180700 1 -- 172.21.15.19:0/2545132049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a4108640 msgr2=0x7fa4a4108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:31.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.876+0000 7fa4aa180700 1 --2- 172.21.15.19:0/2545132049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a4108640 0x7fa4a4108a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa4940098a0 tx=0x7fa494009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:31.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.876+0000 7fa4aa180700 1 -- 172.21.15.19:0/2545132049 shutdown_connections 2024-04-25T11:57:31.680 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.876+0000 7fa4aa180700 1 --2- 172.21.15.19:0/2545132049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a4108640 0x7fa4a4108a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.876+0000 7fa4aa180700 1 -- 172.21.15.19:0/2545132049 >> 172.21.15.19:0/2545132049 conn(0x7fa4a4103c80 msgr2=0x7fa4a41060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.876+0000 7fa4aa180700 1 -- 172.21.15.19:0/2545132049 shutdown_connections 2024-04-25T11:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.876+0000 7fa4aa180700 1 -- 172.21.15.19:0/2545132049 wait complete. 2024-04-25T11:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.877+0000 7fa4aa180700 1 Processor -- start 2024-04-25T11:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.877+0000 7fa4aa180700 1 -- start start 2024-04-25T11:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.877+0000 7fa4aa180700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a4108640 0x7fa4a419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.877+0000 7fa4aa180700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a410cba0 con 0x7fa4a4108640 2024-04-25T11:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.878+0000 7fa4a37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a4108640 0x7fa4a419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.878+0000 7fa4a37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a4108640 0x7fa4a419c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52838/0 (socket says 172.21.15.19:52838) 2024-04-25T11:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.878+0000 7fa4a37fe700 1 -- 172.21.15.19:0/4273379801 learned_addr learned my addr 172.21.15.19:0/4273379801 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.878+0000 7fa4a37fe700 1 -- 172.21.15.19:0/4273379801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a419c900 con 0x7fa4a4108640 2024-04-25T11:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.878+0000 7fa4a37fe700 1 --2- 172.21.15.19:0/4273379801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a4108640 0x7fa4a419c3c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa4940117e0 tx=0x7fa494011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.878+0000 7fa4a0ff9700 1 -- 172.21.15.19:0/4273379801 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa494003b30 con 0x7fa4a4108640 2024-04-25T11:57:31.681 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.878+0000 7fa4aa180700 1 -- 172.21.15.19:0/4273379801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a419cb90 con 0x7fa4a4108640 2024-04-25T11:57:31.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.878+0000 7fa4a0ff9700 1 -- 172.21.15.19:0/4273379801 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7fa494003cd0 con 0x7fa4a4108640 2024-04-25T11:57:31.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.878+0000 7fa4a0ff9700 1 -- 172.21.15.19:0/4273379801 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa49401a830 con 0x7fa4a4108640 2024-04-25T11:57:31.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.878+0000 7fa4aa180700 1 -- 172.21.15.19:0/4273379801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a419cff0 con 0x7fa4a4108640 2024-04-25T11:57:31.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.879+0000 7fa4a0ff9700 1 -- 172.21.15.19:0/4273379801 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa49401a9d0 con 0x7fa4a4108640 2024-04-25T11:57:31.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.879+0000 7fa4aa180700 1 -- 172.21.15.19:0/4273379801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a40640a0 con 0x7fa4a4108640 2024-04-25T11:57:31.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.879+0000 7fa4a0ff9700 1 --2- 172.21.15.19:0/4273379801 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fa48c03da80 0x7fa48c03ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:31.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.879+0000 7fa4a0ff9700 1 -- 172.21.15.19:0/4273379801 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa49404d9e0 con 0x7fa4a4108640 2024-04-25T11:57:31.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.880+0000 7fa4a2ffd700 1 --2- 172.21.15.19:0/4273379801 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fa48c03da80 0x7fa48c03ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:31.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.880+0000 7fa4a2ffd700 1 --2- 172.21.15.19:0/4273379801 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fa48c03da80 0x7fa48c03ff40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa498006fd0 tx=0x7fa498006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:31.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:30.885+0000 7fa4a0ff9700 1 -- 172.21.15.19:0/4273379801 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa494019c70 con 0x7fa4a4108640 2024-04-25T11:57:31.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:31.093+0000 7fa4aa180700 1 -- 172.21.15.19:0/4273379801 --> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fa4a4104d90 con 0x7fa48c03da80 2024-04-25T11:57:31.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:31.094+0000 7fa4a0ff9700 1 -- 172.21.15.19:0/4273379801 <== mgr.14120 v2:172.21.15.19:6800/963006875 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fa4a4104d90 con 0x7fa48c03da80 2024-04-25T11:57:31.682 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:31.096+0000 7fa4aa180700 1 -- 172.21.15.19:0/4273379801 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fa48c03da80 msgr2=0x7fa48c03ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:31.096+0000 7fa4aa180700 1 --2- 172.21.15.19:0/4273379801 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fa48c03da80 0x7fa48c03ff40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa498006fd0 tx=0x7fa498006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:31.096+0000 7fa4aa180700 1 -- 172.21.15.19:0/4273379801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a4108640 msgr2=0x7fa4a419c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:31.096+0000 7fa4aa180700 1 --2- 172.21.15.19:0/4273379801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a4108640 0x7fa4a419c3c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa4940117e0 tx=0x7fa494011ce0 comp rx=0 tx=0).stop 2024-04-25T11:57:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:31.096+0000 7fa4aa180700 1 -- 172.21.15.19:0/4273379801 shutdown_connections 2024-04-25T11:57:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:31.096+0000 7fa4aa180700 1 --2- 172.21.15.19:0/4273379801 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fa48c03da80 0x7fa48c03ff40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:31.096+0000 7fa4aa180700 1 --2- 172.21.15.19:0/4273379801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4a4108640 0x7fa4a419c3c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:31.096+0000 7fa4aa180700 1 -- 172.21.15.19:0/4273379801 >> 172.21.15.19:0/4273379801 conn(0x7fa4a4103c80 msgr2=0x7fa4a4104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:31.096+0000 7fa4aa180700 1 -- 172.21.15.19:0/4273379801 shutdown_connections 2024-04-25T11:57:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:31.096+0000 7fa4aa180700 1 -- 172.21.15.19:0/4273379801 wait complete. 2024-04-25T11:57:31.683 INFO:teuthology.orchestra.run.smithi019.stdout:Generating ssh key... 2024-04-25T11:57:32.777 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: Generating public/private rsa key pair. 2024-04-25T11:57:32.778 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: Your identification has been saved in /tmp/tmp88xjxi7b/key. 2024-04-25T11:57:32.778 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: Your public key has been saved in /tmp/tmp88xjxi7b/key.pub. 2024-04-25T11:57:32.778 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: The key fingerprint is: 2024-04-25T11:57:32.778 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: SHA256:787b3j4lM4yWuulD8Mszbu0rn0jWZtp8cWRCXQ9Prts ceph-be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:57:32.778 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: The key's randomart image is: 2024-04-25T11:57:32.778 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: +---[RSA 3072]----+ 2024-04-25T11:57:32.778 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: | ..+| 2024-04-25T11:57:32.778 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: | .=o| 2024-04-25T11:57:32.778 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: | . +| 2024-04-25T11:57:32.778 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: | . ..o| 2024-04-25T11:57:32.778 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: | So +.+ | 2024-04-25T11:57:32.779 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: | .o.+ *oo| 2024-04-25T11:57:32.779 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: | o+=+ .BE| 2024-04-25T11:57:32.779 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: | =XX.oo | 2024-04-25T11:57:32.779 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: | =%@X+o. | 2024-04-25T11:57:32.779 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: +----[SHA256]-----+ 2024-04-25T11:57:33.066 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.081+0000 7f4b74127700 1 Processor -- start 2024-04-25T11:57:33.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.081+0000 7f4b74127700 1 -- start start 2024-04-25T11:57:33.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.082+0000 7f4b74127700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6c108a60 0x7f4b6c108e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:33.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.082+0000 7f4b74127700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c109450 con 0x7f4b6c108a60 2024-04-25T11:57:33.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.082+0000 7f4b71ec3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6c108a60 0x7f4b6c108e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:33.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.082+0000 7f4b71ec3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6c108a60 0x7f4b6c108e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38098/0 (socket says 172.21.15.19:38098) 2024-04-25T11:57:33.067 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.082+0000 7f4b71ec3700 1 -- 172.21.15.19:0/632910808 learned_addr learned my addr 172.21.15.19:0/632910808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:33.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.082+0000 7f4b71ec3700 1 -- 172.21.15.19:0/632910808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c1095d0 con 0x7f4b6c108a60 2024-04-25T11:57:33.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.082+0000 7f4b71ec3700 1 --2- 172.21.15.19:0/632910808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6c108a60 0x7f4b6c108e80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4b680098a0 tx=0x7f4b68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d165ce57a2ddce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:33.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.082+0000 7f4b70ec1700 1 -- 172.21.15.19:0/632910808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b68004160 con 0x7f4b6c108a60 2024-04-25T11:57:33.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.082+0000 7f4b70ec1700 1 -- 172.21.15.19:0/632910808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f4b68004300 con 0x7f4b6c108a60 2024-04-25T11:57:33.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.082+0000 7f4b70ec1700 1 -- 172.21.15.19:0/632910808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b68011b20 con 0x7f4b6c108a60 2024-04-25T11:57:33.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.083+0000 7f4b74127700 1 -- 172.21.15.19:0/632910808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6c108a60 msgr2=0x7f4b6c108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:33.068 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.083+0000 7f4b74127700 1 --2- 172.21.15.19:0/632910808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6c108a60 0x7f4b6c108e80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4b680098a0 tx=0x7f4b68009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:33.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.083+0000 7f4b74127700 1 -- 172.21.15.19:0/632910808 shutdown_connections 2024-04-25T11:57:33.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.083+0000 7f4b74127700 1 --2- 172.21.15.19:0/632910808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6c108a60 0x7f4b6c108e80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:33.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.083+0000 7f4b74127700 1 -- 172.21.15.19:0/632910808 >> 172.21.15.19:0/632910808 conn(0x7f4b6c1021b0 msgr2=0x7f4b6c1045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:33.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.083+0000 7f4b74127700 1 -- 172.21.15.19:0/632910808 shutdown_connections 2024-04-25T11:57:33.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.083+0000 7f4b74127700 1 -- 172.21.15.19:0/632910808 wait complete. 2024-04-25T11:57:33.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.084+0000 7f4b74127700 1 Processor -- start 2024-04-25T11:57:33.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.084+0000 7f4b74127700 1 -- start start 2024-04-25T11:57:33.069 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.084+0000 7f4b74127700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6c108a60 0x7f4b6c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:33.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.084+0000 7f4b74127700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c107360 con 0x7f4b6c108a60 2024-04-25T11:57:33.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.084+0000 7f4b71ec3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6c108a60 0x7f4b6c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:33.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.084+0000 7f4b71ec3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6c108a60 0x7f4b6c19e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38110/0 (socket says 172.21.15.19:38110) 2024-04-25T11:57:33.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.084+0000 7f4b71ec3700 1 -- 172.21.15.19:0/1000570367 learned_addr learned my addr 172.21.15.19:0/1000570367 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:33.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.084+0000 7f4b71ec3700 1 -- 172.21.15.19:0/1000570367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c19eaa0 con 0x7f4b6c108a60 2024-04-25T11:57:33.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.085+0000 7f4b71ec3700 1 --2- 172.21.15.19:0/1000570367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6c108a60 0x7f4b6c19e560 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4b68006af0 tx=0x7f4b68011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:33.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.085+0000 7f4b5effd700 1 -- 172.21.15.19:0/1000570367 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b680039a0 con 0x7f4b6c108a60 2024-04-25T11:57:33.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.085+0000 7f4b5effd700 1 -- 172.21.15.19:0/1000570367 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f4b68003b40 con 0x7f4b6c108a60 2024-04-25T11:57:33.070 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.085+0000 7f4b74127700 1 -- 172.21.15.19:0/1000570367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6c19ed90 con 0x7f4b6c108a60 2024-04-25T11:57:33.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.085+0000 7f4b5effd700 1 -- 172.21.15.19:0/1000570367 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b68019670 con 0x7f4b6c108a60 2024-04-25T11:57:33.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.085+0000 7f4b74127700 1 -- 172.21.15.19:0/1000570367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b6c1a1a20 con 0x7f4b6c108a60 2024-04-25T11:57:33.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.086+0000 7f4b5effd700 1 -- 172.21.15.19:0/1000570367 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4b6801aa90 con 0x7f4b6c108a60 2024-04-25T11:57:33.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.086+0000 7f4b74127700 1 -- 172.21.15.19:0/1000570367 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6c0640a0 con 0x7f4b6c108a60 2024-04-25T11:57:33.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.086+0000 7f4b5effd700 1 --2- 172.21.15.19:0/1000570367 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f4b58039670 0x7f4b5803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:33.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.086+0000 7f4b5effd700 1 -- 172.21.15.19:0/1000570367 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4b6804d8c0 con 0x7f4b6c108a60 2024-04-25T11:57:33.071 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.086+0000 7f4b716c2700 1 --2- 172.21.15.19:0/1000570367 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f4b58039670 0x7f4b5803bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:33.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.086+0000 7f4b716c2700 1 --2- 172.21.15.19:0/1000570367 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f4b58039670 0x7f4b5803bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4b60006fd0 tx=0x7f4b60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:33.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.091+0000 7f4b5effd700 1 -- 172.21.15.19:0/1000570367 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b68011e20 con 0x7f4b6c108a60 2024-04-25T11:57:33.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.300+0000 7f4b74127700 1 -- 172.21.15.19:0/1000570367 --> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f4b6c062e80 con 0x7f4b58039670 2024-04-25T11:57:33.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.430+0000 7f4b5effd700 1 -- 172.21.15.19:0/1000570367 <== mgr.14120 v2:172.21.15.19:6800/963006875 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4b6c062e80 con 0x7f4b58039670 2024-04-25T11:57:33.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.432+0000 7f4b74127700 1 -- 172.21.15.19:0/1000570367 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f4b58039670 msgr2=0x7f4b5803bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:33.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.432+0000 7f4b74127700 1 --2- 172.21.15.19:0/1000570367 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f4b58039670 0x7f4b5803bb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4b60006fd0 tx=0x7f4b60006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:33.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.432+0000 7f4b74127700 1 -- 172.21.15.19:0/1000570367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6c108a60 msgr2=0x7f4b6c19e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:33.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.432+0000 7f4b74127700 1 --2- 172.21.15.19:0/1000570367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6c108a60 0x7f4b6c19e560 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4b68006af0 tx=0x7f4b68011ce0 comp rx=0 tx=0).stop 2024-04-25T11:57:33.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.432+0000 7f4b74127700 1 -- 172.21.15.19:0/1000570367 shutdown_connections 2024-04-25T11:57:33.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.432+0000 7f4b74127700 1 --2- 172.21.15.19:0/1000570367 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f4b58039670 0x7f4b5803bb30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:33.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.432+0000 7f4b74127700 1 --2- 172.21.15.19:0/1000570367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b6c108a60 0x7f4b6c19e560 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:33.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.432+0000 7f4b74127700 1 -- 172.21.15.19:0/1000570367 >> 172.21.15.19:0/1000570367 conn(0x7f4b6c1021b0 msgr2=0x7f4b6c194ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:33.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.432+0000 7f4b74127700 1 -- 172.21.15.19:0/1000570367 shutdown_connections 2024-04-25T11:57:33.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:32.432+0000 7f4b74127700 1 -- 172.21.15.19:0/1000570367 wait complete. 2024-04-25T11:57:33.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:33 smithi019 ceph-mon[29213]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:57:33.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:33 smithi019 ceph-mon[29213]: Generating ssh key... 2024-04-25T11:57:33.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:33 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' 2024-04-25T11:57:33.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:33 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' 2024-04-25T11:57:34.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6Piaq9f/B27YRiSSrRtQc69+e/F4AyIehtNaz7mPAhfCrjyxWSMlxSEi3q4HX26OJ6ghULMf9eOsk9JwP2Z7vm9UuTEUU0mnoQXuHIpORKqd6azFFQ98wBQQXVGmbwO5pRsHeFziHjDrSJvIoi4wznI4T03ULbjSNJau+e1aBPuOWWzZ8m2jKjeZRnAaK2p46l2ZJO3llMfPVtp/1Ep/QZW1LcB3Az/wbXBDfHNiScld8AwYiLY5ueXDGJB17ZiCt69fD4a86xJJhLk2E7jaHBNNjOgfGDrNfRGM+LzWEMlBqqgg8R2isSTazP45xx/K2AMEl5w78TJkAtl9CdLddbLMDkwJ0XqwIp4XKRqHOeP22luStYJHytIVHEK1e0ZrZJb0+FphAl4mK8id+ued+Z0v930H70wFNMDV9zS9nBzKQ/mPzJxCZfilObQz4kXhDjS0YqG5sycgjzom745EAoSUzkJ0asyyemrIeKADgo0h0a55OsFcqZ3Yh7u8B8qk= ceph-be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:57:34.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.491+0000 7f96009bd700 1 Processor -- start 2024-04-25T11:57:34.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.491+0000 7f96009bd700 1 -- start start 2024-04-25T11:57:34.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.492+0000 7f96009bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc108650 0x7f95fc108a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:34.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.492+0000 7f96009bd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc109040 con 0x7f95fc108650 2024-04-25T11:57:34.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.492+0000 7f95fa59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc108650 0x7f95fc108a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:34.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.492+0000 7f95fa59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc108650 0x7f95fc108a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38112/0 (socket says 172.21.15.19:38112) 2024-04-25T11:57:34.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.492+0000 7f95fa59c700 1 -- 172.21.15.19:0/816228869 learned_addr learned my addr 172.21.15.19:0/816228869 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:34.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.492+0000 7f95fa59c700 1 -- 172.21.15.19:0/816228869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc109800 con 0x7f95fc108650 2024-04-25T11:57:34.276 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.492+0000 7f95fa59c700 1 --2- 172.21.15.19:0/816228869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc108650 0x7f95fc108a70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0098a0 tx=0x7f95ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d81617c1c60cab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:34.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.493+0000 7f95f959a700 1 -- 172.21.15.19:0/816228869 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95ec004160 con 0x7f95fc108650 2024-04-25T11:57:34.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.493+0000 7f95f959a700 1 -- 172.21.15.19:0/816228869 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f95ec004300 con 0x7f95fc108650 2024-04-25T11:57:34.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.493+0000 7f95f959a700 1 -- 172.21.15.19:0/816228869 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95ec011b20 con 0x7f95fc108650 2024-04-25T11:57:34.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.493+0000 7f96009bd700 1 -- 172.21.15.19:0/816228869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc108650 msgr2=0x7f95fc108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:34.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.493+0000 7f96009bd700 1 --2- 172.21.15.19:0/816228869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc108650 0x7f95fc108a70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0098a0 tx=0x7f95ec009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:34.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.493+0000 7f96009bd700 1 -- 172.21.15.19:0/816228869 shutdown_connections 2024-04-25T11:57:34.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.493+0000 7f96009bd700 1 --2- 172.21.15.19:0/816228869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc108650 0x7f95fc108a70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:34.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.493+0000 7f96009bd700 1 -- 172.21.15.19:0/816228869 >> 172.21.15.19:0/816228869 conn(0x7f95fc103cb0 msgr2=0x7f95fc1060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:34.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.493+0000 7f96009bd700 1 -- 172.21.15.19:0/816228869 shutdown_connections 2024-04-25T11:57:34.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.493+0000 7f96009bd700 1 -- 172.21.15.19:0/816228869 wait complete. 2024-04-25T11:57:34.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.494+0000 7f96009bd700 1 Processor -- start 2024-04-25T11:57:34.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.494+0000 7f96009bd700 1 -- start start 2024-04-25T11:57:34.277 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.494+0000 7f96009bd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc108650 0x7f95fc07db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:34.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.494+0000 7f96009bd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc10cbb0 con 0x7f95fc108650 2024-04-25T11:57:34.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.494+0000 7f95fa59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc108650 0x7f95fc07db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:34.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.494+0000 7f95fa59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc108650 0x7f95fc07db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38122/0 (socket says 172.21.15.19:38122) 2024-04-25T11:57:34.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.494+0000 7f95fa59c700 1 -- 172.21.15.19:0/2076956127 learned_addr learned my addr 172.21.15.19:0/2076956127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:34.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.495+0000 7f95fa59c700 1 -- 172.21.15.19:0/2076956127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc07e0a0 con 0x7f95fc108650 2024-04-25T11:57:34.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.495+0000 7f95fa59c700 1 --2- 172.21.15.19:0/2076956127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc108650 0x7f95fc07db60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f95ec006af0 tx=0x7f95ec011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:34.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.495+0000 7f95eb7fe700 1 -- 172.21.15.19:0/2076956127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95ec003b90 con 0x7f95fc108650 2024-04-25T11:57:34.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.495+0000 7f95eb7fe700 1 -- 172.21.15.19:0/2076956127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f95ec003d30 con 0x7f95fc108650 2024-04-25T11:57:34.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.495+0000 7f95eb7fe700 1 -- 172.21.15.19:0/2076956127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95ec0197c0 con 0x7f95fc108650 2024-04-25T11:57:34.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.495+0000 7f96009bd700 1 -- 172.21.15.19:0/2076956127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95fc07c290 con 0x7f95fc108650 2024-04-25T11:57:34.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.495+0000 7f96009bd700 1 -- 172.21.15.19:0/2076956127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95fc07c750 con 0x7f95fc108650 2024-04-25T11:57:34.278 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.496+0000 7f95eb7fe700 1 -- 172.21.15.19:0/2076956127 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f95ec022020 con 0x7f95fc108650 2024-04-25T11:57:34.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.496+0000 7f96009bd700 1 -- 172.21.15.19:0/2076956127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95fc0640a0 con 0x7f95fc108650 2024-04-25T11:57:34.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.496+0000 7f95eb7fe700 1 --2- 172.21.15.19:0/2076956127 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f95e40396c0 0x7f95e403bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:34.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.497+0000 7f95eb7fe700 1 -- 172.21.15.19:0/2076956127 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f95ec04da60 con 0x7f95fc108650 2024-04-25T11:57:34.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.497+0000 7f95f9d9b700 1 --2- 172.21.15.19:0/2076956127 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f95e40396c0 0x7f95e403bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:34.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.497+0000 7f95f9d9b700 1 --2- 172.21.15.19:0/2076956127 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f95e40396c0 0x7f95e403bb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f95f0006fd0 tx=0x7f95f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:34.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.502+0000 7f95eb7fe700 1 -- 172.21.15.19:0/2076956127 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95ec02c720 con 0x7f95fc108650 2024-04-25T11:57:34.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.710+0000 7f96009bd700 1 -- 172.21.15.19:0/2076956127 --> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f95fc07d130 con 0x7f95e40396c0 2024-04-25T11:57:34.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.710+0000 7f95eb7fe700 1 -- 172.21.15.19:0/2076956127 <== mgr.14120 v2:172.21.15.19:6800/963006875 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f95fc07d130 con 0x7f95e40396c0 2024-04-25T11:57:34.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.712+0000 7f96009bd700 1 -- 172.21.15.19:0/2076956127 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f95e40396c0 msgr2=0x7f95e403bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:34.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.712+0000 7f96009bd700 1 --2- 172.21.15.19:0/2076956127 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f95e40396c0 0x7f95e403bb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f95f0006fd0 tx=0x7f95f0006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:34.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.712+0000 7f96009bd700 1 -- 172.21.15.19:0/2076956127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc108650 msgr2=0x7f95fc07db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:34.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.712+0000 7f96009bd700 1 --2- 172.21.15.19:0/2076956127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc108650 0x7f95fc07db60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f95ec006af0 tx=0x7f95ec011ce0 comp rx=0 tx=0).stop 2024-04-25T11:57:34.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.713+0000 7f96009bd700 1 -- 172.21.15.19:0/2076956127 shutdown_connections 2024-04-25T11:57:34.279 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.713+0000 7f96009bd700 1 --2- 172.21.15.19:0/2076956127 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f95e40396c0 0x7f95e403bb80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:34.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.713+0000 7f96009bd700 1 --2- 172.21.15.19:0/2076956127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95fc108650 0x7f95fc07db60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:34.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.713+0000 7f96009bd700 1 -- 172.21.15.19:0/2076956127 >> 172.21.15.19:0/2076956127 conn(0x7f95fc103cb0 msgr2=0x7f95fc1046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:34.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.713+0000 7f96009bd700 1 -- 172.21.15.19:0/2076956127 shutdown_connections 2024-04-25T11:57:34.280 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:33.713+0000 7f96009bd700 1 -- 172.21.15.19:0/2076956127 wait complete. 2024-04-25T11:57:34.280 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T11:57:34.280 INFO:teuthology.orchestra.run.smithi019.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T11:57:34.280 INFO:teuthology.orchestra.run.smithi019.stdout:Adding host smithi019... 2024-04-25T11:57:34.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:34 smithi019 ceph-mon[29213]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:57:35.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:35 smithi019 ceph-mon[29213]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:57:36.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:36 smithi019 ceph-mon[29213]: Deploying cephadm binary to smithi019 2024-04-25T11:57:39.784 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Added host 'smithi019' with addr '172.21.15.19' 2024-04-25T11:57:39.784 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.690+0000 7f187399f700 1 Processor -- start 2024-04-25T11:57:39.784 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.690+0000 7f187399f700 1 -- start start 2024-04-25T11:57:39.784 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.691+0000 7f187399f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f186c109070 0x7f186c10b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:39.784 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.691+0000 7f187399f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f186c07be40 con 0x7f186c109070 2024-04-25T11:57:39.784 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.691+0000 7f187173b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f186c109070 0x7f186c10b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:39.785 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.691+0000 7f187173b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f186c109070 0x7f186c10b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38136/0 (socket says 172.21.15.19:38136) 2024-04-25T11:57:39.785 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.691+0000 7f187173b700 1 -- 172.21.15.19:0/2748225354 learned_addr learned my addr 172.21.15.19:0/2748225354 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:39.785 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.691+0000 7f187173b700 1 -- 172.21.15.19:0/2748225354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c10b9a0 con 0x7f186c109070 2024-04-25T11:57:39.785 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.691+0000 7f187173b700 1 --2- 172.21.15.19:0/2748225354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f186c109070 0x7f186c10b460 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1864009e80 tx=0x7f186400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebe8f04e6464e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:39.785 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.691+0000 7f186bfff700 1 -- 172.21.15.19:0/2748225354 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1864017070 con 0x7f186c109070 2024-04-25T11:57:39.785 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.691+0000 7f186bfff700 1 -- 172.21.15.19:0/2748225354 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f186400b8c0 con 0x7f186c109070 2024-04-25T11:57:39.785 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.692+0000 7f186bfff700 1 -- 172.21.15.19:0/2748225354 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18640045c0 con 0x7f186c109070 2024-04-25T11:57:39.785 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.692+0000 7f187399f700 1 -- 172.21.15.19:0/2748225354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f186c109070 msgr2=0x7f186c10b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:39.785 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.692+0000 7f187399f700 1 --2- 172.21.15.19:0/2748225354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f186c109070 0x7f186c10b460 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1864009e80 tx=0x7f186400b330 comp rx=0 tx=0).stop 2024-04-25T11:57:39.785 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.692+0000 7f187399f700 1 -- 172.21.15.19:0/2748225354 shutdown_connections 2024-04-25T11:57:39.785 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.692+0000 7f187399f700 1 --2- 172.21.15.19:0/2748225354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f186c109070 0x7f186c10b460 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:39.785 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.692+0000 7f187399f700 1 -- 172.21.15.19:0/2748225354 >> 172.21.15.19:0/2748225354 conn(0x7f186c102a40 msgr2=0x7f186c104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:39.786 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.692+0000 7f187399f700 1 -- 172.21.15.19:0/2748225354 shutdown_connections 2024-04-25T11:57:39.786 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.692+0000 7f187399f700 1 -- 172.21.15.19:0/2748225354 wait complete. 2024-04-25T11:57:39.786 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.693+0000 7f187399f700 1 Processor -- start 2024-04-25T11:57:39.786 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.693+0000 7f187399f700 1 -- start start 2024-04-25T11:57:39.786 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.693+0000 7f187399f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f186c109070 0x7f186c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:39.786 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.693+0000 7f187399f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f186c10c970 con 0x7f186c109070 2024-04-25T11:57:39.786 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.693+0000 7f187173b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f186c109070 0x7f186c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:39.786 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.693+0000 7f187173b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f186c109070 0x7f186c19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38150/0 (socket says 172.21.15.19:38150) 2024-04-25T11:57:39.786 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.693+0000 7f187173b700 1 -- 172.21.15.19:0/1063551464 learned_addr learned my addr 172.21.15.19:0/1063551464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:39.786 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.693+0000 7f187173b700 1 -- 172.21.15.19:0/1063551464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c19c990 con 0x7f186c109070 2024-04-25T11:57:39.786 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.694+0000 7f187173b700 1 --2- 172.21.15.19:0/1063551464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f186c109070 0x7f186c19c450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1864000f80 tx=0x7f186400bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:39.786 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.694+0000 7f186a7fc700 1 -- 172.21.15.19:0/1063551464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1864017070 con 0x7f186c109070 2024-04-25T11:57:39.787 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.694+0000 7f186a7fc700 1 -- 172.21.15.19:0/1063551464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f1864022070 con 0x7f186c109070 2024-04-25T11:57:39.787 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.694+0000 7f186a7fc700 1 -- 172.21.15.19:0/1063551464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f186401d6d0 con 0x7f186c109070 2024-04-25T11:57:39.787 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.694+0000 7f187399f700 1 -- 172.21.15.19:0/1063551464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c19cc20 con 0x7f186c109070 2024-04-25T11:57:39.787 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.694+0000 7f187399f700 1 -- 172.21.15.19:0/1063551464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c19f910 con 0x7f186c109070 2024-04-25T11:57:39.787 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.695+0000 7f186a7fc700 1 -- 172.21.15.19:0/1063551464 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1864012a70 con 0x7f186c109070 2024-04-25T11:57:39.787 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.695+0000 7f187399f700 1 -- 172.21.15.19:0/1063551464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186c0640a0 con 0x7f186c109070 2024-04-25T11:57:39.787 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.695+0000 7f186a7fc700 1 --2- 172.21.15.19:0/1063551464 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f18540396c0 0x7f185403bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:39.787 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.695+0000 7f186a7fc700 1 -- 172.21.15.19:0/1063551464 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f186400f050 con 0x7f186c109070 2024-04-25T11:57:39.787 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.695+0000 7f1870f3a700 1 --2- 172.21.15.19:0/1063551464 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f18540396c0 0x7f185403bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:39.787 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.695+0000 7f1870f3a700 1 --2- 172.21.15.19:0/1063551464 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f18540396c0 0x7f185403bb80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1860006fd0 tx=0x7f1860006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:39.787 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.700+0000 7f186a7fc700 1 -- 172.21.15.19:0/1063551464 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1864025360 con 0x7f186c109070 2024-04-25T11:57:39.787 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:34.917+0000 7f187399f700 1 -- 172.21.15.19:0/1063551464 --> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}) -- 0x7f186c103bb0 con 0x7f18540396c0 2024-04-25T11:57:39.787 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:39.162+0000 7f186a7fc700 1 -- 172.21.15.19:0/1063551464 <== mgr.14120 v2:172.21.15.19:6800/963006875 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f186c103bb0 con 0x7f18540396c0 2024-04-25T11:57:39.788 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:39.164+0000 7f187399f700 1 -- 172.21.15.19:0/1063551464 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f18540396c0 msgr2=0x7f185403bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:39.788 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:39.164+0000 7f187399f700 1 --2- 172.21.15.19:0/1063551464 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f18540396c0 0x7f185403bb80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1860006fd0 tx=0x7f1860006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:39.788 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:39.164+0000 7f187399f700 1 -- 172.21.15.19:0/1063551464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f186c109070 msgr2=0x7f186c19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:39.788 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:39.164+0000 7f187399f700 1 --2- 172.21.15.19:0/1063551464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f186c109070 0x7f186c19c450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1864000f80 tx=0x7f186400bdc0 comp rx=0 tx=0).stop 2024-04-25T11:57:39.788 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:39.164+0000 7f187399f700 1 -- 172.21.15.19:0/1063551464 shutdown_connections 2024-04-25T11:57:39.788 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:39.164+0000 7f187399f700 1 --2- 172.21.15.19:0/1063551464 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f18540396c0 0x7f185403bb80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:39.788 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:39.164+0000 7f187399f700 1 --2- 172.21.15.19:0/1063551464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f186c109070 0x7f186c19c450 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:39.788 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:39.164+0000 7f187399f700 1 -- 172.21.15.19:0/1063551464 >> 172.21.15.19:0/1063551464 conn(0x7f186c102a40 msgr2=0x7f186c1034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:39.788 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:39.165+0000 7f187399f700 1 -- 172.21.15.19:0/1063551464 shutdown_connections 2024-04-25T11:57:39.788 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:39.165+0000 7f187399f700 1 -- 172.21.15.19:0/1063551464 wait complete. 2024-04-25T11:57:39.788 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mon service... 2024-04-25T11:57:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:40 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' 2024-04-25T11:57:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:40 smithi019 ceph-mon[29213]: Added host smithi019 2024-04-25T11:57:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:40 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:57:41.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T11:57:41.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.191+0000 7f3b43e5a700 1 Processor -- start 2024-04-25T11:57:41.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.191+0000 7f3b43e5a700 1 -- start start 2024-04-25T11:57:41.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.191+0000 7f3b43e5a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b3c07de00 0x7f3b3c07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:41.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.191+0000 7f3b43e5a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c07c7f0 con 0x7f3b3c07de00 2024-04-25T11:57:41.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.191+0000 7f3b41bf6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b3c07de00 0x7f3b3c07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:41.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.192+0000 7f3b41bf6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b3c07de00 0x7f3b3c07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38152/0 (socket says 172.21.15.19:38152) 2024-04-25T11:57:41.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.192+0000 7f3b41bf6700 1 -- 172.21.15.19:0/268859014 learned_addr learned my addr 172.21.15.19:0/268859014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:41.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.192+0000 7f3b41bf6700 1 -- 172.21.15.19:0/268859014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3c07c970 con 0x7f3b3c07de00 2024-04-25T11:57:41.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.192+0000 7f3b41bf6700 1 --2- 172.21.15.19:0/268859014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b3c07de00 0x7f3b3c07c2b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b380098a0 tx=0x7f3b38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bb6772161f0f40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:41.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.192+0000 7f3b40bf4700 1 -- 172.21.15.19:0/268859014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b38004160 con 0x7f3b3c07de00 2024-04-25T11:57:41.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.192+0000 7f3b40bf4700 1 -- 172.21.15.19:0/268859014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f3b38004300 con 0x7f3b3c07de00 2024-04-25T11:57:41.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.192+0000 7f3b40bf4700 1 -- 172.21.15.19:0/268859014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b38011b20 con 0x7f3b3c07de00 2024-04-25T11:57:41.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.193+0000 7f3b43e5a700 1 -- 172.21.15.19:0/268859014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b3c07de00 msgr2=0x7f3b3c07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:41.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.193+0000 7f3b43e5a700 1 --2- 172.21.15.19:0/268859014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b3c07de00 0x7f3b3c07c2b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b380098a0 tx=0x7f3b38009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:41.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.193+0000 7f3b43e5a700 1 -- 172.21.15.19:0/268859014 shutdown_connections 2024-04-25T11:57:41.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.193+0000 7f3b43e5a700 1 --2- 172.21.15.19:0/268859014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b3c07de00 0x7f3b3c07c2b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:41.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.193+0000 7f3b43e5a700 1 -- 172.21.15.19:0/268859014 >> 172.21.15.19:0/268859014 conn(0x7f3b3c103be0 msgr2=0x7f3b3c106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:41.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.193+0000 7f3b43e5a700 1 -- 172.21.15.19:0/268859014 shutdown_connections 2024-04-25T11:57:41.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.193+0000 7f3b43e5a700 1 -- 172.21.15.19:0/268859014 wait complete. 2024-04-25T11:57:41.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.193+0000 7f3b43e5a700 1 Processor -- start 2024-04-25T11:57:41.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.194+0000 7f3b43e5a700 1 -- start start 2024-04-25T11:57:41.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.194+0000 7f3b43e5a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b3c07de00 0x7f3b3c1a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:41.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.194+0000 7f3b43e5a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3c10ebb0 con 0x7f3b3c07de00 2024-04-25T11:57:41.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.194+0000 7f3b41bf6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b3c07de00 0x7f3b3c1a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:41.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.194+0000 7f3b41bf6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b3c07de00 0x7f3b3c1a29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38156/0 (socket says 172.21.15.19:38156) 2024-04-25T11:57:41.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.194+0000 7f3b41bf6700 1 -- 172.21.15.19:0/831716782 learned_addr learned my addr 172.21.15.19:0/831716782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:41.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.194+0000 7f3b41bf6700 1 -- 172.21.15.19:0/831716782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3c1a2f30 con 0x7f3b3c07de00 2024-04-25T11:57:41.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.194+0000 7f3b41bf6700 1 --2- 172.21.15.19:0/831716782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b3c07de00 0x7f3b3c1a29f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3b380116a0 tx=0x7f3b38011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:41.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.194+0000 7f3b32ffd700 1 -- 172.21.15.19:0/831716782 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b38003ab0 con 0x7f3b3c07de00 2024-04-25T11:57:41.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.194+0000 7f3b43e5a700 1 -- 172.21.15.19:0/831716782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3c1a31c0 con 0x7f3b3c07de00 2024-04-25T11:57:41.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.194+0000 7f3b32ffd700 1 -- 172.21.15.19:0/831716782 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1104+0+0 (secure 0 0 0) 0x7f3b38003c50 con 0x7f3b3c07de00 2024-04-25T11:57:41.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.194+0000 7f3b32ffd700 1 -- 172.21.15.19:0/831716782 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b38019870 con 0x7f3b3c07de00 2024-04-25T11:57:41.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.195+0000 7f3b43e5a700 1 -- 172.21.15.19:0/831716782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b3c1a3620 con 0x7f3b3c07de00 2024-04-25T11:57:41.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.195+0000 7f3b32ffd700 1 -- 172.21.15.19:0/831716782 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3b3801a430 con 0x7f3b3c07de00 2024-04-25T11:57:41.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.195+0000 7f3b43e5a700 1 -- 172.21.15.19:0/831716782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3c0640a0 con 0x7f3b3c07de00 2024-04-25T11:57:41.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.195+0000 7f3b32ffd700 1 --2- 172.21.15.19:0/831716782 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f3b28039620 0x7f3b2803bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:41.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.196+0000 7f3b32ffd700 1 -- 172.21.15.19:0/831716782 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3b3804d8f0 con 0x7f3b3c07de00 2024-04-25T11:57:41.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.196+0000 7f3b413f5700 1 --2- 172.21.15.19:0/831716782 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f3b28039620 0x7f3b2803bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:41.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.196+0000 7f3b413f5700 1 --2- 172.21.15.19:0/831716782 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f3b28039620 0x7f3b2803bae0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c006fd0 tx=0x7f3b2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:41.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.201+0000 7f3b32ffd700 1 -- 172.21.15.19:0/831716782 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b38011e20 con 0x7f3b3c07de00 2024-04-25T11:57:41.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.422+0000 7f3b43e5a700 1 -- 172.21.15.19:0/831716782 --> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3b3c104d30 con 0x7f3b28039620 2024-04-25T11:57:41.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.430+0000 7f3b32ffd700 1 -- 172.21.15.19:0/831716782 <== mgr.14120 v2:172.21.15.19:6800/963006875 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3b3c104d30 con 0x7f3b28039620 2024-04-25T11:57:41.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.432+0000 7f3b43e5a700 1 -- 172.21.15.19:0/831716782 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f3b28039620 msgr2=0x7f3b2803bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:41.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.432+0000 7f3b43e5a700 1 --2- 172.21.15.19:0/831716782 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f3b28039620 0x7f3b2803bae0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c006fd0 tx=0x7f3b2c006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:41.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.432+0000 7f3b43e5a700 1 -- 172.21.15.19:0/831716782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b3c07de00 msgr2=0x7f3b3c1a29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:41.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.432+0000 7f3b43e5a700 1 --2- 172.21.15.19:0/831716782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b3c07de00 0x7f3b3c1a29f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3b380116a0 tx=0x7f3b38011ce0 comp rx=0 tx=0).stop 2024-04-25T11:57:41.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.432+0000 7f3b43e5a700 1 -- 172.21.15.19:0/831716782 shutdown_connections 2024-04-25T11:57:41.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.432+0000 7f3b43e5a700 1 --2- 172.21.15.19:0/831716782 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f3b28039620 0x7f3b2803bae0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:41.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.432+0000 7f3b43e5a700 1 --2- 172.21.15.19:0/831716782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b3c07de00 0x7f3b3c1a29f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:41.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.432+0000 7f3b43e5a700 1 -- 172.21.15.19:0/831716782 >> 172.21.15.19:0/831716782 conn(0x7f3b3c103be0 msgr2=0x7f3b3c104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:41.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.433+0000 7f3b43e5a700 1 -- 172.21.15.19:0/831716782 shutdown_connections 2024-04-25T11:57:41.007 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:40.433+0000 7f3b43e5a700 1 -- 172.21.15.19:0/831716782 wait complete. 2024-04-25T11:57:41.007 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mgr service... 2024-04-25T11:57:41.563 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:41 smithi019 ceph-mon[29213]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:57:41.564 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:41 smithi019 ceph-mon[29213]: Saving service mon spec with placement count:5 2024-04-25T11:57:41.564 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:41 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' 2024-04-25T11:57:43.263 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T11:57:43.263 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.481+0000 7f0dca226700 1 Processor -- start 2024-04-25T11:57:43.263 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.481+0000 7f0dca226700 1 -- start start 2024-04-25T11:57:43.263 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.481+0000 7f0dca226700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc407de80 0x7f0dc407c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:43.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.481+0000 7f0dca226700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc407c900 con 0x7f0dc407de80 2024-04-25T11:57:43.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.481+0000 7f0dc37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc407de80 0x7f0dc407c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:43.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.481+0000 7f0dc37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc407de80 0x7f0dc407c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34190/0 (socket says 172.21.15.19:34190) 2024-04-25T11:57:43.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.481+0000 7f0dc37fe700 1 -- 172.21.15.19:0/3996269810 learned_addr learned my addr 172.21.15.19:0/3996269810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:43.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.482+0000 7f0dc37fe700 1 -- 172.21.15.19:0/3996269810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc407ca80 con 0x7f0dc407de80 2024-04-25T11:57:43.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.482+0000 7f0dc37fe700 1 --2- 172.21.15.19:0/3996269810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc407de80 0x7f0dc407c330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0db40098a0 tx=0x7f0db4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b1dc438b20dce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:43.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.482+0000 7f0dc27fc700 1 -- 172.21.15.19:0/3996269810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0db4004160 con 0x7f0dc407de80 2024-04-25T11:57:43.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.482+0000 7f0dc27fc700 1 -- 172.21.15.19:0/3996269810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0db4004300 con 0x7f0dc407de80 2024-04-25T11:57:43.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.482+0000 7f0dc27fc700 1 -- 172.21.15.19:0/3996269810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0db4011af0 con 0x7f0dc407de80 2024-04-25T11:57:43.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.483+0000 7f0dca226700 1 -- 172.21.15.19:0/3996269810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc407de80 msgr2=0x7f0dc407c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:43.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.483+0000 7f0dca226700 1 --2- 172.21.15.19:0/3996269810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc407de80 0x7f0dc407c330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0db40098a0 tx=0x7f0db4009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:43.264 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.483+0000 7f0dca226700 1 -- 172.21.15.19:0/3996269810 shutdown_connections 2024-04-25T11:57:43.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.483+0000 7f0dca226700 1 --2- 172.21.15.19:0/3996269810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc407de80 0x7f0dc407c330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:43.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.483+0000 7f0dca226700 1 -- 172.21.15.19:0/3996269810 >> 172.21.15.19:0/3996269810 conn(0x7f0dc4103c10 msgr2=0x7f0dc4106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:43.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.483+0000 7f0dca226700 1 -- 172.21.15.19:0/3996269810 shutdown_connections 2024-04-25T11:57:43.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.483+0000 7f0dca226700 1 -- 172.21.15.19:0/3996269810 wait complete. 2024-04-25T11:57:43.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.483+0000 7f0dca226700 1 Processor -- start 2024-04-25T11:57:43.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.483+0000 7f0dca226700 1 -- start start 2024-04-25T11:57:43.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.484+0000 7f0dca226700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc407de80 0x7f0dc419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:43.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.484+0000 7f0dca226700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc410ec30 con 0x7f0dc407de80 2024-04-25T11:57:43.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.484+0000 7f0dc37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc407de80 0x7f0dc419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:43.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.484+0000 7f0dc37fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc407de80 0x7f0dc419e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34192/0 (socket says 172.21.15.19:34192) 2024-04-25T11:57:43.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.484+0000 7f0dc37fe700 1 -- 172.21.15.19:0/2354755015 learned_addr learned my addr 172.21.15.19:0/2354755015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:43.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.484+0000 7f0dc37fe700 1 -- 172.21.15.19:0/2354755015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc419ead0 con 0x7f0dc407de80 2024-04-25T11:57:43.265 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.484+0000 7f0dc37fe700 1 --2- 172.21.15.19:0/2354755015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc407de80 0x7f0dc419e590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0db4003f90 tx=0x7f0db4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:43.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.484+0000 7f0dc0ff9700 1 -- 172.21.15.19:0/2354755015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0db40039e0 con 0x7f0dc407de80 2024-04-25T11:57:43.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.484+0000 7f0dc0ff9700 1 -- 172.21.15.19:0/2354755015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0db4003b80 con 0x7f0dc407de80 2024-04-25T11:57:43.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.484+0000 7f0dca226700 1 -- 172.21.15.19:0/2354755015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc419ed60 con 0x7f0dc407de80 2024-04-25T11:57:43.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.484+0000 7f0dc0ff9700 1 -- 172.21.15.19:0/2354755015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0db40196e0 con 0x7f0dc407de80 2024-04-25T11:57:43.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.484+0000 7f0dca226700 1 -- 172.21.15.19:0/2354755015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc41a1a50 con 0x7f0dc407de80 2024-04-25T11:57:43.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.485+0000 7f0dc0ff9700 1 -- 172.21.15.19:0/2354755015 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0db4022020 con 0x7f0dc407de80 2024-04-25T11:57:43.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.486+0000 7f0dca226700 1 -- 172.21.15.19:0/2354755015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc410edb0 con 0x7f0dc407de80 2024-04-25T11:57:43.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.486+0000 7f0dc0ff9700 1 --2- 172.21.15.19:0/2354755015 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f0dac0396c0 0x7f0dac03bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:43.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.486+0000 7f0dc0ff9700 1 -- 172.21.15.19:0/2354755015 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0db404dad0 con 0x7f0dc407de80 2024-04-25T11:57:43.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.486+0000 7f0dc2ffd700 1 --2- 172.21.15.19:0/2354755015 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f0dac0396c0 0x7f0dac03bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:43.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.486+0000 7f0dc2ffd700 1 --2- 172.21.15.19:0/2354755015 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f0dac0396c0 0x7f0dac03bb80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0db8006fd0 tx=0x7f0db8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:43.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.491+0000 7f0dc0ff9700 1 -- 172.21.15.19:0/2354755015 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0db4029250 con 0x7f0dc407de80 2024-04-25T11:57:43.266 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.711+0000 7f0dca226700 1 -- 172.21.15.19:0/2354755015 --> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0dc4104d80 con 0x7f0dac0396c0 2024-04-25T11:57:43.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.717+0000 7f0dc0ff9700 1 -- 172.21.15.19:0/2354755015 <== mgr.14120 v2:172.21.15.19:6800/963006875 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0dc4104d80 con 0x7f0dac0396c0 2024-04-25T11:57:43.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.719+0000 7f0dca226700 1 -- 172.21.15.19:0/2354755015 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f0dac0396c0 msgr2=0x7f0dac03bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:43.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.719+0000 7f0dca226700 1 --2- 172.21.15.19:0/2354755015 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f0dac0396c0 0x7f0dac03bb80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0db8006fd0 tx=0x7f0db8006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:43.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.719+0000 7f0dca226700 1 -- 172.21.15.19:0/2354755015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc407de80 msgr2=0x7f0dc419e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:43.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.719+0000 7f0dca226700 1 --2- 172.21.15.19:0/2354755015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc407de80 0x7f0dc419e590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0db4003f90 tx=0x7f0db4011b70 comp rx=0 tx=0).stop 2024-04-25T11:57:43.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.719+0000 7f0dca226700 1 -- 172.21.15.19:0/2354755015 shutdown_connections 2024-04-25T11:57:43.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.719+0000 7f0dca226700 1 --2- 172.21.15.19:0/2354755015 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f0dac0396c0 0x7f0dac03bb80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:43.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.719+0000 7f0dca226700 1 --2- 172.21.15.19:0/2354755015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc407de80 0x7f0dc419e590 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:43.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.719+0000 7f0dca226700 1 -- 172.21.15.19:0/2354755015 >> 172.21.15.19:0/2354755015 conn(0x7f0dc4103c10 msgr2=0x7f0dc4104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:43.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.719+0000 7f0dca226700 1 -- 172.21.15.19:0/2354755015 shutdown_connections 2024-04-25T11:57:43.267 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:42.719+0000 7f0dca226700 1 -- 172.21.15.19:0/2354755015 wait complete. 2024-04-25T11:57:43.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:43 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' 2024-04-25T11:57:43.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:43 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' 2024-04-25T11:57:43.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:43 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' 2024-04-25T11:57:44.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:44 smithi019 ceph-mon[29213]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:57:44.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:44 smithi019 ceph-mon[29213]: Saving service mgr spec with placement count:2 2024-04-25T11:57:44.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:44 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2754483098' entity='client.admin' 2024-04-25T11:57:44.771 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.723+0000 7fe9d5376700 1 Processor -- start 2024-04-25T11:57:44.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.723+0000 7fe9d5376700 1 -- start start 2024-04-25T11:57:44.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.723+0000 7fe9d5376700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d01085c0 0x7fe9d01089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:44.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.724+0000 7fe9d5376700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d0108fb0 con 0x7fe9d01085c0 2024-04-25T11:57:44.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.724+0000 7fe9ceffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d01085c0 0x7fe9d01089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:44.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.724+0000 7fe9ceffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d01085c0 0x7fe9d01089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34206/0 (socket says 172.21.15.19:34206) 2024-04-25T11:57:44.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.724+0000 7fe9ceffd700 1 -- 172.21.15.19:0/163528452 learned_addr learned my addr 172.21.15.19:0/163528452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:44.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.724+0000 7fe9ceffd700 1 -- 172.21.15.19:0/163528452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d0109770 con 0x7fe9d01085c0 2024-04-25T11:57:44.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.724+0000 7fe9ceffd700 1 --2- 172.21.15.19:0/163528452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d01085c0 0x7fe9d01089e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00098a0 tx=0x7fe9c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b991a02d29126131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:44.772 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.724+0000 7fe9cdffb700 1 -- 172.21.15.19:0/163528452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9c0004160 con 0x7fe9d01085c0 2024-04-25T11:57:44.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.725+0000 7fe9cdffb700 1 -- 172.21.15.19:0/163528452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe9c0004300 con 0x7fe9d01085c0 2024-04-25T11:57:44.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.725+0000 7fe9cdffb700 1 -- 172.21.15.19:0/163528452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9c0011af0 con 0x7fe9d01085c0 2024-04-25T11:57:44.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.725+0000 7fe9d5376700 1 -- 172.21.15.19:0/163528452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d01085c0 msgr2=0x7fe9d01089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:44.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.725+0000 7fe9d5376700 1 --2- 172.21.15.19:0/163528452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d01085c0 0x7fe9d01089e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00098a0 tx=0x7fe9c0009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:44.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.726+0000 7fe9d5376700 1 -- 172.21.15.19:0/163528452 shutdown_connections 2024-04-25T11:57:44.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.726+0000 7fe9d5376700 1 --2- 172.21.15.19:0/163528452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d01085c0 0x7fe9d01089e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:44.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.726+0000 7fe9d5376700 1 -- 172.21.15.19:0/163528452 >> 172.21.15.19:0/163528452 conn(0x7fe9d0103be0 msgr2=0x7fe9d0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:44.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.726+0000 7fe9d5376700 1 -- 172.21.15.19:0/163528452 shutdown_connections 2024-04-25T11:57:44.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.726+0000 7fe9d5376700 1 -- 172.21.15.19:0/163528452 wait complete. 2024-04-25T11:57:44.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.726+0000 7fe9d5376700 1 Processor -- start 2024-04-25T11:57:44.773 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.727+0000 7fe9d5376700 1 -- start start 2024-04-25T11:57:44.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.727+0000 7fe9d5376700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d01085c0 0x7fe9d019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:44.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.727+0000 7fe9d5376700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d010cb20 con 0x7fe9d01085c0 2024-04-25T11:57:44.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.727+0000 7fe9ceffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d01085c0 0x7fe9d019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:44.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.727+0000 7fe9ceffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d01085c0 0x7fe9d019c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34214/0 (socket says 172.21.15.19:34214) 2024-04-25T11:57:44.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.727+0000 7fe9ceffd700 1 -- 172.21.15.19:0/2754483098 learned_addr learned my addr 172.21.15.19:0/2754483098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:44.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.727+0000 7fe9ceffd700 1 -- 172.21.15.19:0/2754483098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d019c940 con 0x7fe9d01085c0 2024-04-25T11:57:44.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.728+0000 7fe9ceffd700 1 --2- 172.21.15.19:0/2754483098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d01085c0 0x7fe9d019c400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009520 tx=0x7fe9c0003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:44.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.728+0000 7fe9b7fff700 1 -- 172.21.15.19:0/2754483098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9c00036a0 con 0x7fe9d01085c0 2024-04-25T11:57:44.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.728+0000 7fe9d5376700 1 -- 172.21.15.19:0/2754483098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d019cbd0 con 0x7fe9d01085c0 2024-04-25T11:57:44.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.728+0000 7fe9b7fff700 1 -- 172.21.15.19:0/2754483098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe9c001a430 con 0x7fe9d01085c0 2024-04-25T11:57:44.774 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.728+0000 7fe9d5376700 1 -- 172.21.15.19:0/2754483098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d019f8c0 con 0x7fe9d01085c0 2024-04-25T11:57:44.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.728+0000 7fe9b7fff700 1 -- 172.21.15.19:0/2754483098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9c00193c0 con 0x7fe9d01085c0 2024-04-25T11:57:44.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.729+0000 7fe9d5376700 1 -- 172.21.15.19:0/2754483098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9bc005360 con 0x7fe9d01085c0 2024-04-25T11:57:44.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.729+0000 7fe9b7fff700 1 -- 172.21.15.19:0/2754483098 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe9c0019560 con 0x7fe9d01085c0 2024-04-25T11:57:44.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.729+0000 7fe9b7fff700 1 --2- 172.21.15.19:0/2754483098 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fe9b8039670 0x7fe9b803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:44.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.729+0000 7fe9b7fff700 1 -- 172.21.15.19:0/2754483098 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe9c004d870 con 0x7fe9d01085c0 2024-04-25T11:57:44.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.730+0000 7fe9ce7fc700 1 --2- 172.21.15.19:0/2754483098 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fe9b8039670 0x7fe9b803bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:44.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.730+0000 7fe9ce7fc700 1 --2- 172.21.15.19:0/2754483098 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fe9b8039670 0x7fe9b803bb30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4006fd0 tx=0x7fe9c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:44.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.735+0000 7fe9b7fff700 1 -- 172.21.15.19:0/2754483098 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9c000f2e0 con 0x7fe9d01085c0 2024-04-25T11:57:44.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.962+0000 7fe9d5376700 1 -- 172.21.15.19:0/2754483098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fe9bc005650 con 0x7fe9d01085c0 2024-04-25T11:57:44.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.965+0000 7fe9b7fff700 1 -- 172.21.15.19:0/2754483098 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fe9c004d1c0 con 0x7fe9d01085c0 2024-04-25T11:57:44.775 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.968+0000 7fe9d5376700 1 -- 172.21.15.19:0/2754483098 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fe9b8039670 msgr2=0x7fe9b803bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:44.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.968+0000 7fe9d5376700 1 --2- 172.21.15.19:0/2754483098 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fe9b8039670 0x7fe9b803bb30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4006fd0 tx=0x7fe9c4006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:44.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.968+0000 7fe9d5376700 1 -- 172.21.15.19:0/2754483098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d01085c0 msgr2=0x7fe9d019c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:44.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.968+0000 7fe9d5376700 1 --2- 172.21.15.19:0/2754483098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d01085c0 0x7fe9d019c400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009520 tx=0x7fe9c0003eb0 comp rx=0 tx=0).stop 2024-04-25T11:57:44.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.968+0000 7fe9d5376700 1 -- 172.21.15.19:0/2754483098 shutdown_connections 2024-04-25T11:57:44.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.968+0000 7fe9d5376700 1 --2- 172.21.15.19:0/2754483098 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fe9b8039670 0x7fe9b803bb30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:44.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.968+0000 7fe9d5376700 1 --2- 172.21.15.19:0/2754483098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9d01085c0 0x7fe9d019c400 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:44.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.968+0000 7fe9d5376700 1 -- 172.21.15.19:0/2754483098 >> 172.21.15.19:0/2754483098 conn(0x7fe9d0103be0 msgr2=0x7fe9d0104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:44.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.968+0000 7fe9d5376700 1 -- 172.21.15.19:0/2754483098 shutdown_connections 2024-04-25T11:57:44.776 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:43.968+0000 7fe9d5376700 1 -- 172.21.15.19:0/2754483098 wait complete. 2024-04-25T11:57:46.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.573+0000 7ffa6b0ef700 1 Processor -- start 2024-04-25T11:57:46.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.574+0000 7ffa6b0ef700 1 -- start start 2024-04-25T11:57:46.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.574+0000 7ffa6b0ef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa641085b0 0x7ffa641089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:46.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.574+0000 7ffa6b0ef700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa64108fa0 con 0x7ffa641085b0 2024-04-25T11:57:46.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.574+0000 7ffa68e8b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa641085b0 0x7ffa641089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:46.345 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.574+0000 7ffa68e8b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa641085b0 0x7ffa641089d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34230/0 (socket says 172.21.15.19:34230) 2024-04-25T11:57:46.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.574+0000 7ffa68e8b700 1 -- 172.21.15.19:0/2885175008 learned_addr learned my addr 172.21.15.19:0/2885175008 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:46.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.574+0000 7ffa68e8b700 1 -- 172.21.15.19:0/2885175008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa64109760 con 0x7ffa641085b0 2024-04-25T11:57:46.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.574+0000 7ffa68e8b700 1 --2- 172.21.15.19:0/2885175008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa641085b0 0x7ffa641089d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa580098a0 tx=0x7ffa58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b40f474fde0e334 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:46.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.574+0000 7ffa637fe700 1 -- 172.21.15.19:0/2885175008 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa58004160 con 0x7ffa641085b0 2024-04-25T11:57:46.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.575+0000 7ffa637fe700 1 -- 172.21.15.19:0/2885175008 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ffa58004300 con 0x7ffa641085b0 2024-04-25T11:57:46.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.575+0000 7ffa637fe700 1 -- 172.21.15.19:0/2885175008 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa58011af0 con 0x7ffa641085b0 2024-04-25T11:57:46.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.575+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/2885175008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa641085b0 msgr2=0x7ffa641089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:46.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.575+0000 7ffa6b0ef700 1 --2- 172.21.15.19:0/2885175008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa641085b0 0x7ffa641089d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa580098a0 tx=0x7ffa58009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:46.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.575+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/2885175008 shutdown_connections 2024-04-25T11:57:46.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.575+0000 7ffa6b0ef700 1 --2- 172.21.15.19:0/2885175008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa641085b0 0x7ffa641089d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:46.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.575+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/2885175008 >> 172.21.15.19:0/2885175008 conn(0x7ffa64103bb0 msgr2=0x7ffa64105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:46.346 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.575+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/2885175008 shutdown_connections 2024-04-25T11:57:46.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.575+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/2885175008 wait complete. 2024-04-25T11:57:46.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.576+0000 7ffa6b0ef700 1 Processor -- start 2024-04-25T11:57:46.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.576+0000 7ffa6b0ef700 1 -- start start 2024-04-25T11:57:46.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.576+0000 7ffa6b0ef700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa641085b0 0x7ffa6419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:46.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.576+0000 7ffa6b0ef700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6410cb10 con 0x7ffa641085b0 2024-04-25T11:57:46.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.576+0000 7ffa68e8b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa641085b0 0x7ffa6419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:46.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.576+0000 7ffa68e8b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa641085b0 0x7ffa6419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34232/0 (socket says 172.21.15.19:34232) 2024-04-25T11:57:46.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.576+0000 7ffa68e8b700 1 -- 172.21.15.19:0/471716942 learned_addr learned my addr 172.21.15.19:0/471716942 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:46.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.576+0000 7ffa68e8b700 1 -- 172.21.15.19:0/471716942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6419c970 con 0x7ffa641085b0 2024-04-25T11:57:46.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.577+0000 7ffa68e8b700 1 --2- 172.21.15.19:0/471716942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa641085b0 0x7ffa6419c430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa58003f90 tx=0x7ffa58011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:46.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.577+0000 7ffa61ffb700 1 -- 172.21.15.19:0/471716942 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa580039e0 con 0x7ffa641085b0 2024-04-25T11:57:46.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.577+0000 7ffa61ffb700 1 -- 172.21.15.19:0/471716942 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ffa58003b80 con 0x7ffa641085b0 2024-04-25T11:57:46.347 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.577+0000 7ffa61ffb700 1 -- 172.21.15.19:0/471716942 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa580196e0 con 0x7ffa641085b0 2024-04-25T11:57:46.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.577+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/471716942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6419cc00 con 0x7ffa641085b0 2024-04-25T11:57:46.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.577+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/471716942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6419d020 con 0x7ffa641085b0 2024-04-25T11:57:46.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.578+0000 7ffa61ffb700 1 -- 172.21.15.19:0/471716942 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ffa58022020 con 0x7ffa641085b0 2024-04-25T11:57:46.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.578+0000 7ffa61ffb700 1 --2- 172.21.15.19:0/471716942 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7ffa54039720 0x7ffa5403bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:46.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.578+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/471716942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa640640a0 con 0x7ffa641085b0 2024-04-25T11:57:46.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.578+0000 7ffa61ffb700 1 -- 172.21.15.19:0/471716942 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffa5804e920 con 0x7ffa641085b0 2024-04-25T11:57:46.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.578+0000 7ffa63fff700 1 --2- 172.21.15.19:0/471716942 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7ffa54039720 0x7ffa5403bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:46.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.579+0000 7ffa63fff700 1 --2- 172.21.15.19:0/471716942 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7ffa54039720 0x7ffa5403bbe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffa50006fd0 tx=0x7ffa50006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:46.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.584+0000 7ffa61ffb700 1 -- 172.21.15.19:0/471716942 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa58029250 con 0x7ffa641085b0 2024-04-25T11:57:46.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.797+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/471716942 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7ffa6419faf0 con 0x7ffa641085b0 2024-04-25T11:57:46.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.801+0000 7ffa61ffb700 1 -- 172.21.15.19:0/471716942 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7ffa58020e80 con 0x7ffa641085b0 2024-04-25T11:57:46.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.803+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/471716942 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7ffa54039720 msgr2=0x7ffa5403bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:46.348 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.803+0000 7ffa6b0ef700 1 --2- 172.21.15.19:0/471716942 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7ffa54039720 0x7ffa5403bbe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffa50006fd0 tx=0x7ffa50006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:46.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.803+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/471716942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa641085b0 msgr2=0x7ffa6419c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:46.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.803+0000 7ffa6b0ef700 1 --2- 172.21.15.19:0/471716942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa641085b0 0x7ffa6419c430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa58003f90 tx=0x7ffa58011b70 comp rx=0 tx=0).stop 2024-04-25T11:57:46.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.803+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/471716942 shutdown_connections 2024-04-25T11:57:46.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.803+0000 7ffa6b0ef700 1 --2- 172.21.15.19:0/471716942 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7ffa54039720 0x7ffa5403bbe0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:46.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.803+0000 7ffa6b0ef700 1 --2- 172.21.15.19:0/471716942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa641085b0 0x7ffa6419c430 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:46.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.803+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/471716942 >> 172.21.15.19:0/471716942 conn(0x7ffa64103bb0 msgr2=0x7ffa641045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:46.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.804+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/471716942 shutdown_connections 2024-04-25T11:57:46.349 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:45.804+0000 7ffa6b0ef700 1 -- 172.21.15.19:0/471716942 wait complete. 2024-04-25T11:57:46.349 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling the dashboard module... 2024-04-25T11:57:46.756 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:46 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' 2024-04-25T11:57:46.757 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:46 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/471716942' entity='client.admin' 2024-04-25T11:57:47.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:47 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1249922686' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T11:57:47.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:47 smithi019 ceph-mon[29213]: from='mgr.14120 172.21.15.19:0/2515776567' entity='mgr.x' 2024-04-25T11:57:47.994 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:47 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: ignoring --setuser ceph since I am not root 2024-04-25T11:57:47.994 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:47 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: ignoring --setgroup ceph since I am not root 2024-04-25T11:57:47.994 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:47 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:47.830+0000 7efecbc0a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:57:48.292 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:48 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:48.027+0000 7efecbc0a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:57:48.292 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:48 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:48.136+0000 7efecbc0a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:57:48.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.795+0000 7f03737b4700 1 Processor -- start 2024-04-25T11:57:48.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.795+0000 7f03737b4700 1 -- start start 2024-04-25T11:57:48.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.795+0000 7f03737b4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036c1090c0 0x7f036c10b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:48.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.795+0000 7f03737b4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f036c07bea0 con 0x7f036c1090c0 2024-04-25T11:57:48.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.795+0000 7f0371550700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036c1090c0 0x7f036c10b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:48.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.795+0000 7f0371550700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036c1090c0 0x7f036c10b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34236/0 (socket says 172.21.15.19:34236) 2024-04-25T11:57:48.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.795+0000 7f0371550700 1 -- 172.21.15.19:0/1999330049 learned_addr learned my addr 172.21.15.19:0/1999330049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:48.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.796+0000 7f0371550700 1 -- 172.21.15.19:0/1999330049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036c10b9f0 con 0x7f036c1090c0 2024-04-25T11:57:48.302 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.796+0000 7f0371550700 1 --2- 172.21.15.19:0/1999330049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036c1090c0 0x7f036c10b4b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f03680098a0 tx=0x7f0368009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92dc9c1157a5fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.796+0000 7f0363fff700 1 -- 172.21.15.19:0/1999330049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0368004160 con 0x7f036c1090c0 2024-04-25T11:57:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.796+0000 7f0363fff700 1 -- 172.21.15.19:0/1999330049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0368004300 con 0x7f036c1090c0 2024-04-25T11:57:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.796+0000 7f0363fff700 1 -- 172.21.15.19:0/1999330049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0368011af0 con 0x7f036c1090c0 2024-04-25T11:57:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.796+0000 7f03737b4700 1 -- 172.21.15.19:0/1999330049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036c1090c0 msgr2=0x7f036c10b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.796+0000 7f03737b4700 1 --2- 172.21.15.19:0/1999330049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036c1090c0 0x7f036c10b4b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f03680098a0 tx=0x7f0368009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.797+0000 7f03737b4700 1 -- 172.21.15.19:0/1999330049 shutdown_connections 2024-04-25T11:57:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.797+0000 7f03737b4700 1 --2- 172.21.15.19:0/1999330049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036c1090c0 0x7f036c10b4b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.797+0000 7f03737b4700 1 -- 172.21.15.19:0/1999330049 >> 172.21.15.19:0/1999330049 conn(0x7f036c102b30 msgr2=0x7f036c104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.797+0000 7f03737b4700 1 -- 172.21.15.19:0/1999330049 shutdown_connections 2024-04-25T11:57:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.797+0000 7f03737b4700 1 -- 172.21.15.19:0/1999330049 wait complete. 2024-04-25T11:57:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.797+0000 7f03737b4700 1 Processor -- start 2024-04-25T11:57:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.797+0000 7f03737b4700 1 -- start start 2024-04-25T11:57:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.797+0000 7f03737b4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036c1090c0 0x7f036c108940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:48.303 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.797+0000 7f03737b4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f036c10ca60 con 0x7f036c1090c0 2024-04-25T11:57:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.797+0000 7f0371550700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036c1090c0 0x7f036c108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.798+0000 7f0371550700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036c1090c0 0x7f036c108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34238/0 (socket says 172.21.15.19:34238) 2024-04-25T11:57:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.798+0000 7f0371550700 1 -- 172.21.15.19:0/1249922686 learned_addr learned my addr 172.21.15.19:0/1249922686 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.798+0000 7f0371550700 1 -- 172.21.15.19:0/1249922686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036c108e80 con 0x7f036c1090c0 2024-04-25T11:57:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.798+0000 7f0371550700 1 --2- 172.21.15.19:0/1249922686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036c1090c0 0x7f036c108940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f03680117b0 tx=0x7f0368003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.798+0000 7f03627fc700 1 -- 172.21.15.19:0/1249922686 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03680036a0 con 0x7f036c1090c0 2024-04-25T11:57:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.798+0000 7f03627fc700 1 -- 172.21.15.19:0/1249922686 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f036801a430 con 0x7f036c1090c0 2024-04-25T11:57:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.798+0000 7f03737b4700 1 -- 172.21.15.19:0/1249922686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f036c107070 con 0x7f036c1090c0 2024-04-25T11:57:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.798+0000 7f03627fc700 1 -- 172.21.15.19:0/1249922686 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0368019530 con 0x7f036c1090c0 2024-04-25T11:57:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.798+0000 7f03737b4700 1 -- 172.21.15.19:0/1249922686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f036c1075b0 con 0x7f036c1090c0 2024-04-25T11:57:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.799+0000 7f03737b4700 1 -- 172.21.15.19:0/1249922686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f036c0640a0 con 0x7f036c1090c0 2024-04-25T11:57:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.799+0000 7f03627fc700 1 -- 172.21.15.19:0/1249922686 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0368003840 con 0x7f036c1090c0 2024-04-25T11:57:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.799+0000 7f03627fc700 1 --2- 172.21.15.19:0/1249922686 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f0358039620 0x7f035803bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:48.304 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.799+0000 7f03627fc700 1 -- 172.21.15.19:0/1249922686 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f036804d860 con 0x7f036c1090c0 2024-04-25T11:57:48.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.799+0000 7f0370d4f700 1 --2- 172.21.15.19:0/1249922686 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f0358039620 0x7f035803bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:48.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.800+0000 7f0370d4f700 1 --2- 172.21.15.19:0/1249922686 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f0358039620 0x7f035803bae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f035c006fd0 tx=0x7f035c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:48.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:46.804+0000 7f03627fc700 1 -- 172.21.15.19:0/1249922686 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0368003b30 con 0x7f036c1090c0 2024-04-25T11:57:48.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:47.124+0000 7f03737b4700 1 -- 172.21.15.19:0/1249922686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f036c107d00 con 0x7f036c1090c0 2024-04-25T11:57:48.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:47.643+0000 7f03627fc700 1 -- 172.21.15.19:0/1249922686 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f036802cc80 con 0x7f036c1090c0 2024-04-25T11:57:48.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:47.644+0000 7f03627fc700 1 -- 172.21.15.19:0/1249922686 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f03680197e0 con 0x7f036c1090c0 2024-04-25T11:57:48.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:47.646+0000 7f03737b4700 1 -- 172.21.15.19:0/1249922686 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f0358039620 msgr2=0x7f035803bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:48.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:47.646+0000 7f03737b4700 1 --2- 172.21.15.19:0/1249922686 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f0358039620 0x7f035803bae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f035c006fd0 tx=0x7f035c006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:48.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:47.646+0000 7f03737b4700 1 -- 172.21.15.19:0/1249922686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036c1090c0 msgr2=0x7f036c108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:48.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:47.646+0000 7f03737b4700 1 --2- 172.21.15.19:0/1249922686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036c1090c0 0x7f036c108940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f03680117b0 tx=0x7f0368003eb0 comp rx=0 tx=0).stop 2024-04-25T11:57:48.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:47.646+0000 7f03737b4700 1 -- 172.21.15.19:0/1249922686 shutdown_connections 2024-04-25T11:57:48.305 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:47.646+0000 7f03737b4700 1 --2- 172.21.15.19:0/1249922686 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f0358039620 0x7f035803bae0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:48.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:47.646+0000 7f03737b4700 1 --2- 172.21.15.19:0/1249922686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f036c1090c0 0x7f036c108940 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:48.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:47.646+0000 7f03737b4700 1 -- 172.21.15.19:0/1249922686 >> 172.21.15.19:0/1249922686 conn(0x7f036c102b30 msgr2=0x7f036c1034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:48.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:47.646+0000 7f03737b4700 1 -- 172.21.15.19:0/1249922686 shutdown_connections 2024-04-25T11:57:48.306 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:47.646+0000 7f03737b4700 1 -- 172.21.15.19:0/1249922686 wait complete. 2024-04-25T11:57:48.545 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:48 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:48.460+0000 7efecbc0a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:57:48.950 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:48 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1249922686' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T11:57:48.951 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:48 smithi019 ceph-mon[29213]: mgrmap e9: x(active, since 20s) 2024-04-25T11:57:48.952 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:48 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:48.552+0000 7efecbc0a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:57:49.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:49 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/814858311' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T11:57:49.995 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:49 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:49.824+0000 7efecbc0a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:57:49.995 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:50 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:50.001+0000 7efecbc0a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:57:50.266 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:50 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:50.180+0000 7efecbc0a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:57:50.542 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:50 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:50.482+0000 7efecbc0a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:57:50.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:57:50.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T11:57:50.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T11:57:50.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T11:57:50.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T11:57:50.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:57:50.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.751+0000 7fcda40e3700 1 Processor -- start 2024-04-25T11:57:50.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.751+0000 7fcda40e3700 1 -- start start 2024-04-25T11:57:50.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.751+0000 7fcda40e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd9c106f10 0x7fcd9c107330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:50.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.751+0000 7fcda40e3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c07be00 con 0x7fcd9c106f10 2024-04-25T11:57:50.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.751+0000 7fcda1e7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd9c106f10 0x7fcd9c107330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:50.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.751+0000 7fcda1e7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd9c106f10 0x7fcd9c107330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34270/0 (socket says 172.21.15.19:34270) 2024-04-25T11:57:50.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.751+0000 7fcda1e7f700 1 -- 172.21.15.19:0/2033220620 learned_addr learned my addr 172.21.15.19:0/2033220620 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:50.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.751+0000 7fcda1e7f700 1 -- 172.21.15.19:0/2033220620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c107870 con 0x7fcd9c106f10 2024-04-25T11:57:50.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.752+0000 7fcda1e7f700 1 --2- 172.21.15.19:0/2033220620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd9c106f10 0x7fcd9c107330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd980098a0 tx=0x7fcd98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616dca7358570a6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:50.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.752+0000 7fcda0e7d700 1 -- 172.21.15.19:0/2033220620 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd98004160 con 0x7fcd9c106f10 2024-04-25T11:57:50.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.752+0000 7fcda0e7d700 1 -- 172.21.15.19:0/2033220620 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcd98004300 con 0x7fcd9c106f10 2024-04-25T11:57:50.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.752+0000 7fcda0e7d700 1 -- 172.21.15.19:0/2033220620 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd98011af0 con 0x7fcd9c106f10 2024-04-25T11:57:50.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.752+0000 7fcda40e3700 1 -- 172.21.15.19:0/2033220620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd9c106f10 msgr2=0x7fcd9c107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:50.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.752+0000 7fcda40e3700 1 --2- 172.21.15.19:0/2033220620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd9c106f10 0x7fcd9c107330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd980098a0 tx=0x7fcd98009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:50.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.753+0000 7fcda40e3700 1 -- 172.21.15.19:0/2033220620 shutdown_connections 2024-04-25T11:57:50.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.753+0000 7fcda40e3700 1 --2- 172.21.15.19:0/2033220620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd9c106f10 0x7fcd9c107330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:50.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.753+0000 7fcda40e3700 1 -- 172.21.15.19:0/2033220620 >> 172.21.15.19:0/2033220620 conn(0x7fcd9c102ad0 msgr2=0x7fcd9c104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:50.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.753+0000 7fcda40e3700 1 -- 172.21.15.19:0/2033220620 shutdown_connections 2024-04-25T11:57:50.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.753+0000 7fcda40e3700 1 -- 172.21.15.19:0/2033220620 wait complete. 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.753+0000 7fcda40e3700 1 Processor -- start 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.754+0000 7fcda40e3700 1 -- start start 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.754+0000 7fcda40e3700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd9c106f10 0x7fcd9c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.754+0000 7fcda40e3700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c108940 con 0x7fcd9c106f10 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.754+0000 7fcda1e7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd9c106f10 0x7fcd9c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.754+0000 7fcda1e7f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd9c106f10 0x7fcd9c19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34272/0 (socket says 172.21.15.19:34272) 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.754+0000 7fcda1e7f700 1 -- 172.21.15.19:0/814858311 learned_addr learned my addr 172.21.15.19:0/814858311 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.754+0000 7fcda1e7f700 1 -- 172.21.15.19:0/814858311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c19ebd0 con 0x7fcd9c106f10 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.754+0000 7fcda1e7f700 1 --2- 172.21.15.19:0/814858311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd9c106f10 0x7fcd9c19e690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcd980117b0 tx=0x7fcd98011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.754+0000 7fcd92ffd700 1 -- 172.21.15.19:0/814858311 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd980039e0 con 0x7fcd9c106f10 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.754+0000 7fcd92ffd700 1 -- 172.21.15.19:0/814858311 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcd98003b80 con 0x7fcd9c106f10 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.754+0000 7fcd92ffd700 1 -- 172.21.15.19:0/814858311 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd98024780 con 0x7fcd9c106f10 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.754+0000 7fcda40e3700 1 -- 172.21.15.19:0/814858311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9c19ee60 con 0x7fcd9c106f10 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.755+0000 7fcda40e3700 1 -- 172.21.15.19:0/814858311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9c1a1b50 con 0x7fcd9c106f10 2024-04-25T11:57:50.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.755+0000 7fcd92ffd700 1 -- 172.21.15.19:0/814858311 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fcd98022020 con 0x7fcd9c106f10 2024-04-25T11:57:50.596 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.756+0000 7fcda40e3700 1 -- 172.21.15.19:0/814858311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9c0640a0 con 0x7fcd9c106f10 2024-04-25T11:57:50.596 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.756+0000 7fcd92ffd700 1 --2- 172.21.15.19:0/814858311 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fcd88039720 0x7fcd8803bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:50.596 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.756+0000 7fcd92ffd700 1 -- 172.21.15.19:0/814858311 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcd9804d700 con 0x7fcd9c106f10 2024-04-25T11:57:50.596 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.756+0000 7fcda167e700 1 -- 172.21.15.19:0/814858311 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fcd88039720 msgr2=0x7fcd8803bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/963006875 2024-04-25T11:57:50.596 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.756+0000 7fcda167e700 1 --2- 172.21.15.19:0/814858311 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fcd88039720 0x7fcd8803bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:57:50.596 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.761+0000 7fcd92ffd700 1 -- 172.21.15.19:0/814858311 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd98024920 con 0x7fcd9c106f10 2024-04-25T11:57:50.596 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.957+0000 7fcda167e700 1 -- 172.21.15.19:0/814858311 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fcd88039720 msgr2=0x7fcd8803bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/963006875 2024-04-25T11:57:50.596 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:48.957+0000 7fcda167e700 1 --2- 172.21.15.19:0/814858311 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fcd88039720 0x7fcd8803bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:57:50.596 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:49.088+0000 7fcda40e3700 1 -- 172.21.15.19:0/814858311 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcd9c02e1d0 con 0x7fcd9c106f10 2024-04-25T11:57:50.596 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:49.089+0000 7fcd92ffd700 1 -- 172.21.15.19:0/814858311 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fcd9802c350 con 0x7fcd9c106f10 2024-04-25T11:57:50.596 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:49.091+0000 7fcda40e3700 1 -- 172.21.15.19:0/814858311 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fcd88039720 msgr2=0x7fcd8803bbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:57:50.596 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:49.091+0000 7fcda40e3700 1 --2- 172.21.15.19:0/814858311 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fcd88039720 0x7fcd8803bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:50.596 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:49.091+0000 7fcda40e3700 1 -- 172.21.15.19:0/814858311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd9c106f10 msgr2=0x7fcd9c19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:50.596 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:49.091+0000 7fcda40e3700 1 --2- 172.21.15.19:0/814858311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd9c106f10 0x7fcd9c19e690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcd980117b0 tx=0x7fcd98011b70 comp rx=0 tx=0).stop 2024-04-25T11:57:50.597 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:49.091+0000 7fcda40e3700 1 -- 172.21.15.19:0/814858311 shutdown_connections 2024-04-25T11:57:50.597 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:49.091+0000 7fcda40e3700 1 --2- 172.21.15.19:0/814858311 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7fcd88039720 0x7fcd8803bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:50.597 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:49.091+0000 7fcda40e3700 1 --2- 172.21.15.19:0/814858311 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd9c106f10 0x7fcd9c19e690 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:50.597 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:49.091+0000 7fcda40e3700 1 -- 172.21.15.19:0/814858311 >> 172.21.15.19:0/814858311 conn(0x7fcd9c102ad0 msgr2=0x7fcd9c1952d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:50.597 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:49.091+0000 7fcda40e3700 1 -- 172.21.15.19:0/814858311 shutdown_connections 2024-04-25T11:57:50.597 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:49.091+0000 7fcda40e3700 1 -- 172.21.15.19:0/814858311 wait complete. 2024-04-25T11:57:50.597 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-04-25T11:57:50.597 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 9... 2024-04-25T11:57:50.894 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:50 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:50.578+0000 7efecbc0a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:57:50.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:50 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:50.794+0000 7efecbc0a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:57:51.203 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:51 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:51.001+0000 7efecbc0a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:57:52.042 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:51 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:51.627+0000 7efecbc0a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:57:52.370 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:52 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:52.065+0000 7efecbc0a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:57:52.370 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:52 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:52.165+0000 7efecbc0a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:57:52.370 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:52 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:52.376+0000 7efecbc0a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:57:53.267 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:52 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:52.924+0000 7efecbc0a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:57:53.267 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:53 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:53.020+0000 7efecbc0a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:57:53.542 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:53 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:53.273+0000 7efecbc0a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:57:53.542 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:53 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:53.412+0000 7efecbc0a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:57:53.542 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:53 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:53.512+0000 7efecbc0a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:57:53.942 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:53 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:53.633+0000 7efecbc0a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:57:53.942 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:53 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:53.948+0000 7efecbc0a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:57:54.793 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:54.581+0000 7efecbc0a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:57:55.292 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 11:57:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T11:57:54.809+0000 7efecbc0a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:57:55.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:54 smithi019 ceph-mon[29213]: Active manager daemon x restarted 2024-04-25T11:57:55.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:54 smithi019 ceph-mon[29213]: Activating manager daemon x 2024-04-25T11:57:55.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:54 smithi019 ceph-mon[29213]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T11:57:55.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:54 smithi019 ceph-mon[29213]: mgrmap e10: x(active, starting, since 0.00335381s) 2024-04-25T11:57:55.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:57:55.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T11:57:55.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T11:57:55.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T11:57:55.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T11:57:55.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:54 smithi019 ceph-mon[29213]: Manager daemon x is now available 2024-04-25T11:57:55.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:57:56.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T11:57:56.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T11:57:56.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:55 smithi019 ceph-mon[29213]: [25/Apr/2024:11:57:55] ENGINE Bus STARTING 2024-04-25T11:57:56.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:55 smithi019 ceph-mon[29213]: [25/Apr/2024:11:57:55] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T11:57:56.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:55 smithi019 ceph-mon[29213]: [25/Apr/2024:11:57:55] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T11:57:56.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:55 smithi019 ceph-mon[29213]: [25/Apr/2024:11:57:55] ENGINE Bus STARTED 2024-04-25T11:57:56.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:55 smithi019 ceph-mon[29213]: mgrmap e11: x(active, since 1.00591s) 2024-04-25T11:57:56.401 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:57:56.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T11:57:56.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T11:57:56.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:57:56.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.004+0000 7f6ce23d7700 1 Processor -- start 2024-04-25T11:57:56.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.004+0000 7f6ce23d7700 1 -- start start 2024-04-25T11:57:56.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.004+0000 7f6ce23d7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc07de40 0x7f6cdc07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:56.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.004+0000 7f6ce23d7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc07c8c0 con 0x7f6cdc07de40 2024-04-25T11:57:56.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.004+0000 7f6cdbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc07de40 0x7f6cdc07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:56.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.004+0000 7f6cdbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc07de40 0x7f6cdc07c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34286/0 (socket says 172.21.15.19:34286) 2024-04-25T11:57:56.402 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.004+0000 7f6cdbfff700 1 -- 172.21.15.19:0/256017795 learned_addr learned my addr 172.21.15.19:0/256017795 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:56.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.005+0000 7f6cdbfff700 1 -- 172.21.15.19:0/256017795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc07ca40 con 0x7f6cdc07de40 2024-04-25T11:57:56.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.005+0000 7f6cdbfff700 1 --2- 172.21.15.19:0/256017795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc07de40 0x7f6cdc07c2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0098a0 tx=0x7f6ccc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa57242fd4d0601 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:56.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.005+0000 7f6cdaffd700 1 -- 172.21.15.19:0/256017795 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ccc004160 con 0x7f6cdc07de40 2024-04-25T11:57:56.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.005+0000 7f6cdaffd700 1 -- 172.21.15.19:0/256017795 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6ccc004300 con 0x7f6cdc07de40 2024-04-25T11:57:56.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.005+0000 7f6cdaffd700 1 -- 172.21.15.19:0/256017795 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ccc011af0 con 0x7f6cdc07de40 2024-04-25T11:57:56.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.006+0000 7f6ce23d7700 1 -- 172.21.15.19:0/256017795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc07de40 msgr2=0x7f6cdc07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:56.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.006+0000 7f6ce23d7700 1 --2- 172.21.15.19:0/256017795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc07de40 0x7f6cdc07c2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0098a0 tx=0x7f6ccc009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:56.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.006+0000 7f6ce23d7700 1 -- 172.21.15.19:0/256017795 shutdown_connections 2024-04-25T11:57:56.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.006+0000 7f6ce23d7700 1 --2- 172.21.15.19:0/256017795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc07de40 0x7f6cdc07c2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:56.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.006+0000 7f6ce23d7700 1 -- 172.21.15.19:0/256017795 >> 172.21.15.19:0/256017795 conn(0x7f6cdc103bb0 msgr2=0x7f6cdc105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:56.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.006+0000 7f6ce23d7700 1 -- 172.21.15.19:0/256017795 shutdown_connections 2024-04-25T11:57:56.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.006+0000 7f6ce23d7700 1 -- 172.21.15.19:0/256017795 wait complete. 2024-04-25T11:57:56.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.006+0000 7f6ce23d7700 1 Processor -- start 2024-04-25T11:57:56.403 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.006+0000 7f6ce23d7700 1 -- start start 2024-04-25T11:57:56.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.007+0000 7f6ce23d7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc07de40 0x7f6cdc19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:56.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.007+0000 7f6ce23d7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc10eb90 con 0x7f6cdc07de40 2024-04-25T11:57:56.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.007+0000 7f6cdbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc07de40 0x7f6cdc19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:56.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.007+0000 7f6cdbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc07de40 0x7f6cdc19e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34290/0 (socket says 172.21.15.19:34290) 2024-04-25T11:57:56.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.007+0000 7f6cdbfff700 1 -- 172.21.15.19:0/1044648664 learned_addr learned my addr 172.21.15.19:0/1044648664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:56.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.007+0000 7f6cdbfff700 1 -- 172.21.15.19:0/1044648664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc19eb10 con 0x7f6cdc07de40 2024-04-25T11:57:56.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.007+0000 7f6cdbfff700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc07de40 0x7f6cdc19e5d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc003f90 tx=0x7f6ccc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:56.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.008+0000 7f6cd97fa700 1 -- 172.21.15.19:0/1044648664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ccc0039e0 con 0x7f6cdc07de40 2024-04-25T11:57:56.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.008+0000 7f6cd97fa700 1 -- 172.21.15.19:0/1044648664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6ccc003b80 con 0x7f6cdc07de40 2024-04-25T11:57:56.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.008+0000 7f6ce23d7700 1 -- 172.21.15.19:0/1044648664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cdc19eda0 con 0x7f6cdc07de40 2024-04-25T11:57:56.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.008+0000 7f6cd97fa700 1 -- 172.21.15.19:0/1044648664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ccc0196e0 con 0x7f6cdc07de40 2024-04-25T11:57:56.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.008+0000 7f6ce23d7700 1 -- 172.21.15.19:0/1044648664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cdc1a1a90 con 0x7f6cdc07de40 2024-04-25T11:57:56.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.009+0000 7f6cd97fa700 1 -- 172.21.15.19:0/1044648664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f6ccc022020 con 0x7f6cdc07de40 2024-04-25T11:57:56.404 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.009+0000 7f6cd97fa700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f6cc40396c0 0x7f6cc403bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:56.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.009+0000 7f6cd97fa700 1 -- 172.21.15.19:0/1044648664 --> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6cc403c290 con 0x7f6cc40396c0 2024-04-25T11:57:56.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.009+0000 7f6cd97fa700 1 -- 172.21.15.19:0/1044648664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6ccc04d870 con 0x7f6cdc07de40 2024-04-25T11:57:56.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.009+0000 7f6cdb7fe700 1 -- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f6cc40396c0 msgr2=0x7f6cc403bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/963006875 2024-04-25T11:57:56.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.009+0000 7f6cdb7fe700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f6cc40396c0 0x7f6cc403bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:57:56.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.210+0000 7f6cdb7fe700 1 -- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f6cc40396c0 msgr2=0x7f6cc403bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/963006875 2024-04-25T11:57:56.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.210+0000 7f6cdb7fe700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f6cc40396c0 0x7f6cc403bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:57:56.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.611+0000 7f6cdb7fe700 1 -- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f6cc40396c0 msgr2=0x7f6cc403bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/963006875 2024-04-25T11:57:56.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:51.611+0000 7f6cdb7fe700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f6cc40396c0 0x7f6cc403bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:57:56.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:52.412+0000 7f6cdb7fe700 1 -- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f6cc40396c0 msgr2=0x7f6cc403bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/963006875 2024-04-25T11:57:56.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:52.412+0000 7f6cdb7fe700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f6cc40396c0 0x7f6cc403bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:57:56.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:54.013+0000 7f6cdb7fe700 1 -- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f6cc40396c0 msgr2=0x7f6cc403bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/963006875 2024-04-25T11:57:56.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:54.013+0000 7f6cdb7fe700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f6cc40396c0 0x7f6cc403bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T11:57:56.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:54.812+0000 7f6cd97fa700 1 -- 172.21.15.19:0/1044648664 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f6ccc019ce0 con 0x7f6cdc07de40 2024-04-25T11:57:56.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:54.812+0000 7f6cd97fa700 1 -- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f6cc40396c0 msgr2=0x7f6cc403bb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:57:56.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:54.812+0000 7f6cd97fa700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f6cc40396c0 0x7f6cc403bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:56.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.814+0000 7f6cd97fa700 1 -- 172.21.15.19:0/1044648664 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f6ccc04f700 con 0x7f6cdc07de40 2024-04-25T11:57:56.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.814+0000 7f6cd97fa700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6cc403cfb0 0x7f6cc403f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:56.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.814+0000 7f6cd97fa700 1 -- 172.21.15.19:0/1044648664 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6ccc01a610 con 0x7f6cc403cfb0 2024-04-25T11:57:56.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.815+0000 7f6cdb7fe700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6cc403cfb0 0x7f6cc403f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:56.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.815+0000 7f6cdb7fe700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6cc403cfb0 0x7f6cc403f3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0003a10 tx=0x7f6cd00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:56.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.816+0000 7f6cd97fa700 1 -- 172.21.15.19:0/1044648664 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f6ccc01a610 con 0x7f6cc403cfb0 2024-04-25T11:57:56.406 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.822+0000 7f6ce23d7700 1 -- 172.21.15.19:0/1044648664 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6cdc0640a0 con 0x7f6cc403cfb0 2024-04-25T11:57:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.823+0000 7f6cd97fa700 1 -- 172.21.15.19:0/1044648664 <== mgr.14152 v2:172.21.15.19:6800/2853466745 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f6cdc0640a0 con 0x7f6cc403cfb0 2024-04-25T11:57:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.823+0000 7f6ce23d7700 1 -- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6cc403cfb0 msgr2=0x7f6cc403f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.823+0000 7f6ce23d7700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6cc403cfb0 0x7f6cc403f3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0003a10 tx=0x7f6cd00092b0 comp rx=0 tx=0).stop 2024-04-25T11:57:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.823+0000 7f6ce23d7700 1 -- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc07de40 msgr2=0x7f6cdc19e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.823+0000 7f6ce23d7700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc07de40 0x7f6cdc19e5d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc003f90 tx=0x7f6ccc011b70 comp rx=0 tx=0).stop 2024-04-25T11:57:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.823+0000 7f6ce23d7700 1 -- 172.21.15.19:0/1044648664 shutdown_connections 2024-04-25T11:57:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.823+0000 7f6ce23d7700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6cc403cfb0 0x7f6cc403f3a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.823+0000 7f6ce23d7700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cdc07de40 0x7f6cdc19e5d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.823+0000 7f6ce23d7700 1 --2- 172.21.15.19:0/1044648664 >> [v2:172.21.15.19:6800/963006875,v1:172.21.15.19:6801/963006875] conn(0x7f6cc40396c0 0x7f6cc403bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.823+0000 7f6ce23d7700 1 -- 172.21.15.19:0/1044648664 >> 172.21.15.19:0/1044648664 conn(0x7f6cdc103bb0 msgr2=0x7f6cdc1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.823+0000 7f6ce23d7700 1 -- 172.21.15.19:0/1044648664 shutdown_connections 2024-04-25T11:57:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:55.823+0000 7f6ce23d7700 1 -- 172.21.15.19:0/1044648664 wait complete. 2024-04-25T11:57:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 9 is available 2024-04-25T11:57:56.407 INFO:teuthology.orchestra.run.smithi019.stdout:Generating a dashboard self-signed certificate... 2024-04-25T11:57:57.029 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:56 smithi019 ceph-mon[29213]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T11:57:57.029 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:56 smithi019 ceph-mon[29213]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T11:57:57.029 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:57:57.995 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T11:57:57.995 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.064+0000 7f9380857700 1 Processor -- start 2024-04-25T11:57:57.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.064+0000 7f9380857700 1 -- start start 2024-04-25T11:57:57.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.064+0000 7f9380857700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c07de40 0x7f937c07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:57.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.064+0000 7f9380857700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f937c07c8c0 con 0x7f937c07de40 2024-04-25T11:57:57.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.065+0000 7f937a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c07de40 0x7f937c07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:57.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.065+0000 7f937a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c07de40 0x7f937c07c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48582/0 (socket says 172.21.15.19:48582) 2024-04-25T11:57:57.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.065+0000 7f937a59c700 1 -- 172.21.15.19:0/3453638286 learned_addr learned my addr 172.21.15.19:0/3453638286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:57.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.065+0000 7f937a59c700 1 -- 172.21.15.19:0/3453638286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c07ca40 con 0x7f937c07de40 2024-04-25T11:57:57.996 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.065+0000 7f937a59c700 1 --2- 172.21.15.19:0/3453638286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c07de40 0x7f937c07c2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f936c0098a0 tx=0x7f936c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a53bbf01560d217 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:58.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.065+0000 7f937959a700 1 -- 172.21.15.19:0/3453638286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f936c004160 con 0x7f937c07de40 2024-04-25T11:57:58.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.066+0000 7f937959a700 1 -- 172.21.15.19:0/3453638286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f936c004300 con 0x7f937c07de40 2024-04-25T11:57:58.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.066+0000 7f937959a700 1 -- 172.21.15.19:0/3453638286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f936c011af0 con 0x7f937c07de40 2024-04-25T11:57:58.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.066+0000 7f9380857700 1 -- 172.21.15.19:0/3453638286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c07de40 msgr2=0x7f937c07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:58.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.066+0000 7f9380857700 1 --2- 172.21.15.19:0/3453638286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c07de40 0x7f937c07c2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f936c0098a0 tx=0x7f936c009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:58.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.066+0000 7f9380857700 1 -- 172.21.15.19:0/3453638286 shutdown_connections 2024-04-25T11:57:58.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.066+0000 7f9380857700 1 --2- 172.21.15.19:0/3453638286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c07de40 0x7f937c07c2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:58.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.066+0000 7f9380857700 1 -- 172.21.15.19:0/3453638286 >> 172.21.15.19:0/3453638286 conn(0x7f937c103bb0 msgr2=0x7f937c105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:58.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.066+0000 7f9380857700 1 -- 172.21.15.19:0/3453638286 shutdown_connections 2024-04-25T11:57:58.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.066+0000 7f9380857700 1 -- 172.21.15.19:0/3453638286 wait complete. 2024-04-25T11:57:58.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.067+0000 7f9380857700 1 Processor -- start 2024-04-25T11:57:58.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.067+0000 7f9380857700 1 -- start start 2024-04-25T11:57:58.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.067+0000 7f9380857700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c07de40 0x7f937c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:58.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.067+0000 7f9380857700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f937c10eb90 con 0x7f937c07de40 2024-04-25T11:57:58.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.068+0000 7f937a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c07de40 0x7f937c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:58.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.068+0000 7f937a59c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c07de40 0x7f937c19e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48594/0 (socket says 172.21.15.19:48594) 2024-04-25T11:57:58.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.068+0000 7f937a59c700 1 -- 172.21.15.19:0/3326226812 learned_addr learned my addr 172.21.15.19:0/3326226812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:58.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.068+0000 7f937a59c700 1 -- 172.21.15.19:0/3326226812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c19eae0 con 0x7f937c07de40 2024-04-25T11:57:58.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.068+0000 7f937a59c700 1 --2- 172.21.15.19:0/3326226812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c07de40 0x7f937c19e5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f936c003f90 tx=0x7f936c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:58.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.068+0000 7f936b7fe700 1 -- 172.21.15.19:0/3326226812 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f936c0039e0 con 0x7f937c07de40 2024-04-25T11:57:58.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.068+0000 7f9380857700 1 -- 172.21.15.19:0/3326226812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937c19ed70 con 0x7f937c07de40 2024-04-25T11:57:58.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.068+0000 7f936b7fe700 1 -- 172.21.15.19:0/3326226812 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f936c003b80 con 0x7f937c07de40 2024-04-25T11:57:58.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.068+0000 7f936b7fe700 1 -- 172.21.15.19:0/3326226812 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f936c0196e0 con 0x7f937c07de40 2024-04-25T11:57:58.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.068+0000 7f9380857700 1 -- 172.21.15.19:0/3326226812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937c1a1a60 con 0x7f937c07de40 2024-04-25T11:57:58.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.069+0000 7f9380857700 1 -- 172.21.15.19:0/3326226812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f937c0640a0 con 0x7f937c07de40 2024-04-25T11:57:58.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.070+0000 7f936b7fe700 1 -- 172.21.15.19:0/3326226812 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f936c022020 con 0x7f937c07de40 2024-04-25T11:57:58.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.070+0000 7f936b7fe700 1 --2- 172.21.15.19:0/3326226812 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f93640395a0 0x7f936403ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:58.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.070+0000 7f936b7fe700 1 -- 172.21.15.19:0/3326226812 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f936c04d8b0 con 0x7f937c07de40 2024-04-25T11:57:58.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.070+0000 7f9379d9b700 1 --2- 172.21.15.19:0/3326226812 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f93640395a0 0x7f936403ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:58.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.070+0000 7f9379d9b700 1 --2- 172.21.15.19:0/3326226812 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f93640395a0 0x7f936403ba60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9370006fd0 tx=0x7f9370006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:58.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.079+0000 7f936b7fe700 1 -- 172.21.15.19:0/3326226812 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f936c01abf0 con 0x7f937c07de40 2024-04-25T11:57:58.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.294+0000 7f9380857700 1 -- 172.21.15.19:0/3326226812 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f937c104cc0 con 0x7f93640395a0 2024-04-25T11:57:58.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.395+0000 7f936b7fe700 1 -- 172.21.15.19:0/3326226812 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f937c104cc0 con 0x7f93640395a0 2024-04-25T11:57:58.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.398+0000 7f9380857700 1 -- 172.21.15.19:0/3326226812 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f93640395a0 msgr2=0x7f936403ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:58.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.398+0000 7f9380857700 1 --2- 172.21.15.19:0/3326226812 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f93640395a0 0x7f936403ba60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9370006fd0 tx=0x7f9370006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:58.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.398+0000 7f9380857700 1 -- 172.21.15.19:0/3326226812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c07de40 msgr2=0x7f937c19e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:58.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.398+0000 7f9380857700 1 --2- 172.21.15.19:0/3326226812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c07de40 0x7f937c19e5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f936c003f90 tx=0x7f936c011b70 comp rx=0 tx=0).stop 2024-04-25T11:57:58.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.398+0000 7f9380857700 1 -- 172.21.15.19:0/3326226812 shutdown_connections 2024-04-25T11:57:58.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.398+0000 7f9380857700 1 --2- 172.21.15.19:0/3326226812 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f93640395a0 0x7f936403ba60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:58.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.398+0000 7f9380857700 1 --2- 172.21.15.19:0/3326226812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c07de40 0x7f937c19e5a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:58.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.398+0000 7f9380857700 1 -- 172.21.15.19:0/3326226812 >> 172.21.15.19:0/3326226812 conn(0x7f937c103bb0 msgr2=0x7f937c1045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:58.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.398+0000 7f9380857700 1 -- 172.21.15.19:0/3326226812 shutdown_connections 2024-04-25T11:57:58.006 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:57.398+0000 7f9380857700 1 -- 172.21.15.19:0/3326226812 wait complete. 2024-04-25T11:57:58.006 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial admin user... 2024-04-25T11:57:58.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:58 smithi019 ceph-mon[29213]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:57:58.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:57:58.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:57:58.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:58 smithi019 ceph-mon[29213]: mgrmap e12: x(active, since 2s) 2024-04-25T11:57:58.587 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:57:59.635 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$UqYKijiGJWgmUBLY4KtMoe2Xt5sBL2skJtFn.B/NsQo4NuTTTmhDG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714046278, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T11:57:59.635 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.438+0000 7fdbbb0bf700 1 Processor -- start 2024-04-25T11:57:59.635 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.438+0000 7fdbbb0bf700 1 -- start start 2024-04-25T11:57:59.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.438+0000 7fdbbb0bf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb4106e40 0x7fdbb4107260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:59.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.438+0000 7fdbbb0bf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb407be60 con 0x7fdbb4106e40 2024-04-25T11:57:59.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.438+0000 7fdbb8e5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb4106e40 0x7fdbb4107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:59.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.438+0000 7fdbb8e5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb4106e40 0x7fdbb4107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48600/0 (socket says 172.21.15.19:48600) 2024-04-25T11:57:59.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.438+0000 7fdbb8e5b700 1 -- 172.21.15.19:0/3711702633 learned_addr learned my addr 172.21.15.19:0/3711702633 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:59.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.439+0000 7fdbb8e5b700 1 -- 172.21.15.19:0/3711702633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb41077a0 con 0x7fdbb4106e40 2024-04-25T11:57:59.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.439+0000 7fdbb8e5b700 1 --2- 172.21.15.19:0/3711702633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb4106e40 0x7fdbb4107260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdba80098a0 tx=0x7fdba8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17a9580e7f3b210 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:59.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.439+0000 7fdbb37fe700 1 -- 172.21.15.19:0/3711702633 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba8004160 con 0x7fdbb4106e40 2024-04-25T11:57:59.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.439+0000 7fdbb37fe700 1 -- 172.21.15.19:0/3711702633 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdba8004300 con 0x7fdbb4106e40 2024-04-25T11:57:59.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.439+0000 7fdbb37fe700 1 -- 172.21.15.19:0/3711702633 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba8011af0 con 0x7fdbb4106e40 2024-04-25T11:57:59.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.440+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3711702633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb4106e40 msgr2=0x7fdbb4107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:59.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.440+0000 7fdbbb0bf700 1 --2- 172.21.15.19:0/3711702633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb4106e40 0x7fdbb4107260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdba80098a0 tx=0x7fdba8009bb0 comp rx=0 tx=0).stop 2024-04-25T11:57:59.636 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.440+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3711702633 shutdown_connections 2024-04-25T11:57:59.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.440+0000 7fdbbb0bf700 1 --2- 172.21.15.19:0/3711702633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb4106e40 0x7fdbb4107260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:59.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.440+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3711702633 >> 172.21.15.19:0/3711702633 conn(0x7fdbb4102a40 msgr2=0x7fdbb4104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:59.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.440+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3711702633 shutdown_connections 2024-04-25T11:57:59.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.440+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3711702633 wait complete. 2024-04-25T11:57:59.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.441+0000 7fdbbb0bf700 1 Processor -- start 2024-04-25T11:57:59.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.441+0000 7fdbbb0bf700 1 -- start start 2024-04-25T11:57:59.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.441+0000 7fdbbb0bf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb4106e40 0x7fdbb419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:59.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.441+0000 7fdbbb0bf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4108870 con 0x7fdbb4106e40 2024-04-25T11:57:59.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.441+0000 7fdbb8e5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb4106e40 0x7fdbb419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:59.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.441+0000 7fdbb8e5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb4106e40 0x7fdbb419e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48604/0 (socket says 172.21.15.19:48604) 2024-04-25T11:57:59.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.441+0000 7fdbb8e5b700 1 -- 172.21.15.19:0/3936024650 learned_addr learned my addr 172.21.15.19:0/3936024650 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:57:59.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.441+0000 7fdbb8e5b700 1 -- 172.21.15.19:0/3936024650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb419eba0 con 0x7fdbb4106e40 2024-04-25T11:57:59.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.441+0000 7fdbb8e5b700 1 --2- 172.21.15.19:0/3936024650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb4106e40 0x7fdbb419e660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdba8003f90 tx=0x7fdba8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:59.637 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.441+0000 7fdbb1ffb700 1 -- 172.21.15.19:0/3936024650 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba80039e0 con 0x7fdbb4106e40 2024-04-25T11:57:59.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.441+0000 7fdbb1ffb700 1 -- 172.21.15.19:0/3936024650 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdba8003b80 con 0x7fdbb4106e40 2024-04-25T11:57:59.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.442+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3936024650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb419ee30 con 0x7fdbb4106e40 2024-04-25T11:57:59.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.442+0000 7fdbb1ffb700 1 -- 172.21.15.19:0/3936024650 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba80196e0 con 0x7fdbb4106e40 2024-04-25T11:57:59.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.442+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3936024650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb41a1b20 con 0x7fdbb4106e40 2024-04-25T11:57:59.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.443+0000 7fdbb1ffb700 1 -- 172.21.15.19:0/3936024650 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fdba8022020 con 0x7fdbb4106e40 2024-04-25T11:57:59.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.443+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3936024650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb40640a0 con 0x7fdbb4106e40 2024-04-25T11:57:59.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.443+0000 7fdbb1ffb700 1 --2- 172.21.15.19:0/3936024650 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fdba4039720 0x7fdba403bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:57:59.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.443+0000 7fdbb1ffb700 1 -- 172.21.15.19:0/3936024650 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdba804d990 con 0x7fdbb4106e40 2024-04-25T11:57:59.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.443+0000 7fdbb3fff700 1 --2- 172.21.15.19:0/3936024650 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fdba4039720 0x7fdba403bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:57:59.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.444+0000 7fdbb3fff700 1 --2- 172.21.15.19:0/3936024650 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fdba4039720 0x7fdba403bbe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdba0006fd0 tx=0x7fdba0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:57:59.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.449+0000 7fdbb1ffb700 1 -- 172.21.15.19:0/3936024650 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdba8020e80 con 0x7fdbb4106e40 2024-04-25T11:57:59.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.691+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3936024650 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fdbb402e1d0 con 0x7fdba4039720 2024-04-25T11:57:59.638 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.938+0000 7fdbb1ffb700 1 -- 172.21.15.19:0/3936024650 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fdbb402e1d0 con 0x7fdba4039720 2024-04-25T11:57:59.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.940+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3936024650 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fdba4039720 msgr2=0x7fdba403bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:59.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.940+0000 7fdbbb0bf700 1 --2- 172.21.15.19:0/3936024650 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fdba4039720 0x7fdba403bbe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdba0006fd0 tx=0x7fdba0006e40 comp rx=0 tx=0).stop 2024-04-25T11:57:59.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.940+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3936024650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb4106e40 msgr2=0x7fdbb419e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:57:59.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.940+0000 7fdbbb0bf700 1 --2- 172.21.15.19:0/3936024650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb4106e40 0x7fdbb419e660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdba8003f90 tx=0x7fdba8011b70 comp rx=0 tx=0).stop 2024-04-25T11:57:59.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.940+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3936024650 shutdown_connections 2024-04-25T11:57:59.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.940+0000 7fdbbb0bf700 1 --2- 172.21.15.19:0/3936024650 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fdba4039720 0x7fdba403bbe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:59.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.940+0000 7fdbbb0bf700 1 --2- 172.21.15.19:0/3936024650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbb4106e40 0x7fdbb419e660 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:57:59.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.940+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3936024650 >> 172.21.15.19:0/3936024650 conn(0x7fdbb4102a40 msgr2=0x7fdbb4195300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:57:59.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.940+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3936024650 shutdown_connections 2024-04-25T11:57:59.639 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:57:58.940+0000 7fdbbb0bf700 1 -- 172.21.15.19:0/3936024650 wait complete. 2024-04-25T11:57:59.639 INFO:teuthology.orchestra.run.smithi019.stdout:Fetching dashboard port number... 2024-04-25T11:57:59.736 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:59 smithi019 ceph-mon[29213]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:57:59.736 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:57:59 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:00.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:00 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3215248164' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T11:58:01.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T11:58:01.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.056+0000 7f73b86b6700 1 Processor -- start 2024-04-25T11:58:01.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.056+0000 7f73b86b6700 1 -- start start 2024-04-25T11:58:01.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.057+0000 7f73b86b6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b007de20 0x7f73b007c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:01.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.057+0000 7f73b86b6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73b007c810 con 0x7f73b007de20 2024-04-25T11:58:01.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.057+0000 7f73b6452700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b007de20 0x7f73b007c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:01.079 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.057+0000 7f73b6452700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b007de20 0x7f73b007c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48620/0 (socket says 172.21.15.19:48620) 2024-04-25T11:58:01.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.057+0000 7f73b6452700 1 -- 172.21.15.19:0/4265281804 learned_addr learned my addr 172.21.15.19:0/4265281804 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:58:01.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.057+0000 7f73b6452700 1 -- 172.21.15.19:0/4265281804 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b007c990 con 0x7f73b007de20 2024-04-25T11:58:01.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.057+0000 7f73b6452700 1 --2- 172.21.15.19:0/4265281804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b007de20 0x7f73b007c2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0098a0 tx=0x7f73ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89305f1ce8a9ada6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:01.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.057+0000 7f73b5450700 1 -- 172.21.15.19:0/4265281804 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73ac004160 con 0x7f73b007de20 2024-04-25T11:58:01.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.058+0000 7f73b5450700 1 -- 172.21.15.19:0/4265281804 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f73ac004300 con 0x7f73b007de20 2024-04-25T11:58:01.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.058+0000 7f73b5450700 1 -- 172.21.15.19:0/4265281804 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73ac011af0 con 0x7f73b007de20 2024-04-25T11:58:01.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.058+0000 7f73b86b6700 1 -- 172.21.15.19:0/4265281804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b007de20 msgr2=0x7f73b007c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:01.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.058+0000 7f73b86b6700 1 --2- 172.21.15.19:0/4265281804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b007de20 0x7f73b007c2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0098a0 tx=0x7f73ac009bb0 comp rx=0 tx=0).stop 2024-04-25T11:58:01.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.058+0000 7f73b86b6700 1 -- 172.21.15.19:0/4265281804 shutdown_connections 2024-04-25T11:58:01.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.058+0000 7f73b86b6700 1 --2- 172.21.15.19:0/4265281804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b007de20 0x7f73b007c2d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:01.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.058+0000 7f73b86b6700 1 -- 172.21.15.19:0/4265281804 >> 172.21.15.19:0/4265281804 conn(0x7f73b0103bb0 msgr2=0x7f73b0105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:01.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.058+0000 7f73b86b6700 1 -- 172.21.15.19:0/4265281804 shutdown_connections 2024-04-25T11:58:01.080 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.058+0000 7f73b86b6700 1 -- 172.21.15.19:0/4265281804 wait complete. 2024-04-25T11:58:01.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.059+0000 7f73b86b6700 1 Processor -- start 2024-04-25T11:58:01.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.059+0000 7f73b86b6700 1 -- start start 2024-04-25T11:58:01.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.059+0000 7f73b86b6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b007de20 0x7f73b019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:01.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.059+0000 7f73b86b6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73b010eb50 con 0x7f73b007de20 2024-04-25T11:58:01.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.059+0000 7f73b6452700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b007de20 0x7f73b019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:01.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.059+0000 7f73b6452700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b007de20 0x7f73b019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48624/0 (socket says 172.21.15.19:48624) 2024-04-25T11:58:01.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.059+0000 7f73b6452700 1 -- 172.21.15.19:0/3215248164 learned_addr learned my addr 172.21.15.19:0/3215248164 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:58:01.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.060+0000 7f73b6452700 1 -- 172.21.15.19:0/3215248164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b019eb00 con 0x7f73b007de20 2024-04-25T11:58:01.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.060+0000 7f73b6452700 1 --2- 172.21.15.19:0/3215248164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b007de20 0x7f73b019e5c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0117b0 tx=0x7f73ac004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:01.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.060+0000 7f73a77fe700 1 -- 172.21.15.19:0/3215248164 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73ac0036a0 con 0x7f73b007de20 2024-04-25T11:58:01.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.060+0000 7f73a77fe700 1 -- 172.21.15.19:0/3215248164 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f73ac01a430 con 0x7f73b007de20 2024-04-25T11:58:01.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.060+0000 7f73b86b6700 1 -- 172.21.15.19:0/3215248164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b019ed90 con 0x7f73b007de20 2024-04-25T11:58:01.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.060+0000 7f73a77fe700 1 -- 172.21.15.19:0/3215248164 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73ac019530 con 0x7f73b007de20 2024-04-25T11:58:01.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.060+0000 7f73b86b6700 1 -- 172.21.15.19:0/3215248164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b01a1a80 con 0x7f73b007de20 2024-04-25T11:58:01.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.061+0000 7f73a77fe700 1 -- 172.21.15.19:0/3215248164 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f73ac003840 con 0x7f73b007de20 2024-04-25T11:58:01.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.061+0000 7f73b86b6700 1 -- 172.21.15.19:0/3215248164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b00640a0 con 0x7f73b007de20 2024-04-25T11:58:01.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.061+0000 7f73a77fe700 1 --2- 172.21.15.19:0/3215248164 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f739c0396d0 0x7f739c03bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:01.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.061+0000 7f73a77fe700 1 -- 172.21.15.19:0/3215248164 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f73ac04d990 con 0x7f73b007de20 2024-04-25T11:58:01.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.061+0000 7f73b5c51700 1 --2- 172.21.15.19:0/3215248164 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f739c0396d0 0x7f739c03bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:01.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.062+0000 7f73b5c51700 1 --2- 172.21.15.19:0/3215248164 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f739c0396d0 0x7f739c03bb90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f73a0006fd0 tx=0x7f73a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:01.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.066+0000 7f73a77fe700 1 -- 172.21.15.19:0/3215248164 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73ac016df0 con 0x7f73b007de20 2024-04-25T11:58:01.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.275+0000 7f73b86b6700 1 -- 172.21.15.19:0/3215248164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f73b01a2030 con 0x7f73b007de20 2024-04-25T11:58:01.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.275+0000 7f73a77fe700 1 -- 172.21.15.19:0/3215248164 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f73ac0196d0 con 0x7f73b007de20 2024-04-25T11:58:01.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.277+0000 7f73b86b6700 1 -- 172.21.15.19:0/3215248164 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f739c0396d0 msgr2=0x7f739c03bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:01.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.277+0000 7f73b86b6700 1 --2- 172.21.15.19:0/3215248164 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f739c0396d0 0x7f739c03bb90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f73a0006fd0 tx=0x7f73a0006e40 comp rx=0 tx=0).stop 2024-04-25T11:58:01.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.277+0000 7f73b86b6700 1 -- 172.21.15.19:0/3215248164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b007de20 msgr2=0x7f73b019e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:01.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.277+0000 7f73b86b6700 1 --2- 172.21.15.19:0/3215248164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b007de20 0x7f73b019e5c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0117b0 tx=0x7f73ac004610 comp rx=0 tx=0).stop 2024-04-25T11:58:01.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.277+0000 7f73b86b6700 1 -- 172.21.15.19:0/3215248164 shutdown_connections 2024-04-25T11:58:01.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.277+0000 7f73b86b6700 1 --2- 172.21.15.19:0/3215248164 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f739c0396d0 0x7f739c03bb90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:01.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.277+0000 7f73b86b6700 1 --2- 172.21.15.19:0/3215248164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73b007de20 0x7f73b019e5c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:01.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.277+0000 7f73b86b6700 1 -- 172.21.15.19:0/3215248164 >> 172.21.15.19:0/3215248164 conn(0x7f73b0103bb0 msgr2=0x7f73b0104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:01.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.278+0000 7f73b86b6700 1 -- 172.21.15.19:0/3215248164 shutdown_connections 2024-04-25T11:58:01.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:00.278+0000 7f73b86b6700 1 -- 172.21.15.19:0/3215248164 wait complete. 2024-04-25T11:58:01.088 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T11:58:01.088 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-04-25T11:58:01.096 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T11:58:01.096 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-04-25T11:58:01.096 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-04-25T11:58:01.096 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T11:58:01.098 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph Dashboard is now available at: 2024-04-25T11:58:01.098 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:58:01.098 INFO:teuthology.orchestra.run.smithi019.stdout: URL: https://smithi019:8443/ 2024-04-25T11:58:01.098 INFO:teuthology.orchestra.run.smithi019.stdout: User: admin 2024-04-25T11:58:01.098 INFO:teuthology.orchestra.run.smithi019.stdout: Password: 82txb9chts 2024-04-25T11:58:01.098 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:58:01.099 INFO:teuthology.orchestra.run.smithi019.stdout:Saving cluster configuration to /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config directory 2024-04-25T11:58:01.099 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling autotune for osd_memory_target 2024-04-25T11:58:02.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.504+0000 7fd4798f4700 1 Processor -- start 2024-04-25T11:58:02.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.504+0000 7fd4798f4700 1 -- start start 2024-04-25T11:58:02.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.504+0000 7fd4798f4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474108650 0x7fd474108a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:02.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.504+0000 7fd4798f4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd474109040 con 0x7fd474108650 2024-04-25T11:58:02.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.505+0000 7fd472ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474108650 0x7fd474108a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:02.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.505+0000 7fd472ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474108650 0x7fd474108a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48640/0 (socket says 172.21.15.19:48640) 2024-04-25T11:58:02.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.505+0000 7fd472ffd700 1 -- 172.21.15.19:0/8587842 learned_addr learned my addr 172.21.15.19:0/8587842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:58:02.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.505+0000 7fd472ffd700 1 -- 172.21.15.19:0/8587842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd474109800 con 0x7fd474108650 2024-04-25T11:58:02.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.505+0000 7fd472ffd700 1 --2- 172.21.15.19:0/8587842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474108650 0x7fd474108a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009e80 tx=0x7fd46c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb0081213064b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:02.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.505+0000 7fd471ffb700 1 -- 172.21.15.19:0/8587842 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd46c017070 con 0x7fd474108650 2024-04-25T11:58:02.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.505+0000 7fd471ffb700 1 -- 172.21.15.19:0/8587842 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd46c00b8c0 con 0x7fd474108650 2024-04-25T11:58:02.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.505+0000 7fd471ffb700 1 -- 172.21.15.19:0/8587842 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd46c004500 con 0x7fd474108650 2024-04-25T11:58:02.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.506+0000 7fd4798f4700 1 -- 172.21.15.19:0/8587842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474108650 msgr2=0x7fd474108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:02.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.506+0000 7fd4798f4700 1 --2- 172.21.15.19:0/8587842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474108650 0x7fd474108a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009e80 tx=0x7fd46c00b330 comp rx=0 tx=0).stop 2024-04-25T11:58:02.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.506+0000 7fd4798f4700 1 -- 172.21.15.19:0/8587842 shutdown_connections 2024-04-25T11:58:02.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.506+0000 7fd4798f4700 1 --2- 172.21.15.19:0/8587842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474108650 0x7fd474108a70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:02.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.506+0000 7fd4798f4700 1 -- 172.21.15.19:0/8587842 >> 172.21.15.19:0/8587842 conn(0x7fd474103cb0 msgr2=0x7fd4741060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:02.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.506+0000 7fd4798f4700 1 -- 172.21.15.19:0/8587842 shutdown_connections 2024-04-25T11:58:02.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.506+0000 7fd4798f4700 1 -- 172.21.15.19:0/8587842 wait complete. 2024-04-25T11:58:02.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.507+0000 7fd4798f4700 1 Processor -- start 2024-04-25T11:58:02.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.507+0000 7fd4798f4700 1 -- start start 2024-04-25T11:58:02.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.507+0000 7fd4798f4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474108650 0x7fd47407db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:02.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.507+0000 7fd4798f4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd47410cbb0 con 0x7fd474108650 2024-04-25T11:58:02.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.507+0000 7fd472ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474108650 0x7fd47407db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:02.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.507+0000 7fd472ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474108650 0x7fd47407db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48646/0 (socket says 172.21.15.19:48646) 2024-04-25T11:58:02.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.507+0000 7fd472ffd700 1 -- 172.21.15.19:0/3361492277 learned_addr learned my addr 172.21.15.19:0/3361492277 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:58:02.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.507+0000 7fd472ffd700 1 -- 172.21.15.19:0/3361492277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47407e0a0 con 0x7fd474108650 2024-04-25T11:58:02.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.508+0000 7fd472ffd700 1 --2- 172.21.15.19:0/3361492277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474108650 0x7fd47407db60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd46c000f80 tx=0x7fd46c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:02.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.508+0000 7fd4788f2700 1 -- 172.21.15.19:0/3361492277 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd46c021020 con 0x7fd474108650 2024-04-25T11:58:02.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.508+0000 7fd4798f4700 1 -- 172.21.15.19:0/3361492277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47407c290 con 0x7fd474108650 2024-04-25T11:58:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.508+0000 7fd4788f2700 1 -- 172.21.15.19:0/3361492277 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd46c003b50 con 0x7fd474108650 2024-04-25T11:58:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.508+0000 7fd4788f2700 1 -- 172.21.15.19:0/3361492277 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd46c017040 con 0x7fd474108650 2024-04-25T11:58:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.508+0000 7fd4798f4700 1 -- 172.21.15.19:0/3361492277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd47407c7d0 con 0x7fd474108650 2024-04-25T11:58:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.509+0000 7fd4788f2700 1 -- 172.21.15.19:0/3361492277 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd46c024020 con 0x7fd474108650 2024-04-25T11:58:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.509+0000 7fd4798f4700 1 -- 172.21.15.19:0/3361492277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4740640a0 con 0x7fd474108650 2024-04-25T11:58:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.509+0000 7fd4788f2700 1 --2- 172.21.15.19:0/3361492277 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd460039760 0x7fd46003bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.509+0000 7fd4788f2700 1 -- 172.21.15.19:0/3361492277 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd46c00f050 con 0x7fd474108650 2024-04-25T11:58:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.509+0000 7fd4727fc700 1 --2- 172.21.15.19:0/3361492277 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd460039760 0x7fd46003bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.509+0000 7fd4727fc700 1 --2- 172.21.15.19:0/3361492277 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd460039760 0x7fd46003bc20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd45c006fd0 tx=0x7fd45c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.514+0000 7fd4788f2700 1 -- 172.21.15.19:0/3361492277 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd46c02b020 con 0x7fd474108650 2024-04-25T11:58:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.719+0000 7fd4798f4700 1 -- 172.21.15.19:0/3361492277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fd47407cb10 con 0x7fd474108650 2024-04-25T11:58:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.720+0000 7fd4788f2700 1 -- 172.21.15.19:0/3361492277 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fd47407cb10 con 0x7fd474108650 2024-04-25T11:58:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.722+0000 7fd4798f4700 1 -- 172.21.15.19:0/3361492277 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd460039760 msgr2=0x7fd46003bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.722+0000 7fd4798f4700 1 --2- 172.21.15.19:0/3361492277 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd460039760 0x7fd46003bc20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd45c006fd0 tx=0x7fd45c006e40 comp rx=0 tx=0).stop 2024-04-25T11:58:02.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.722+0000 7fd4798f4700 1 -- 172.21.15.19:0/3361492277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474108650 msgr2=0x7fd47407db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:02.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.722+0000 7fd4798f4700 1 --2- 172.21.15.19:0/3361492277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474108650 0x7fd47407db60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd46c000f80 tx=0x7fd46c00bd50 comp rx=0 tx=0).stop 2024-04-25T11:58:02.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.722+0000 7fd4798f4700 1 -- 172.21.15.19:0/3361492277 shutdown_connections 2024-04-25T11:58:02.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.722+0000 7fd4798f4700 1 --2- 172.21.15.19:0/3361492277 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd460039760 0x7fd46003bc20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:02.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.722+0000 7fd4798f4700 1 --2- 172.21.15.19:0/3361492277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd474108650 0x7fd47407db60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:02.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.722+0000 7fd4798f4700 1 -- 172.21.15.19:0/3361492277 >> 172.21.15.19:0/3361492277 conn(0x7fd474103cb0 msgr2=0x7fd4741046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:02.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.722+0000 7fd4798f4700 1 -- 172.21.15.19:0/3361492277 shutdown_connections 2024-04-25T11:58:02.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:01.722+0000 7fd4798f4700 1 -- 172.21.15.19:0/3361492277 wait complete. 2024-04-25T11:58:02.498 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:02 smithi019 ceph-mon[29213]: mgrmap e13: x(active, since 6s) 2024-04-25T11:58:03.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.706+0000 7fa9fe039700 1 Processor -- start 2024-04-25T11:58:03.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.706+0000 7fa9fe039700 1 -- start start 2024-04-25T11:58:03.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.706+0000 7fa9fe039700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f81074a0 0x7fa9f81078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:03.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.706+0000 7fa9fe039700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f8107e90 con 0x7fa9f81074a0 2024-04-25T11:58:03.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.707+0000 7fa9f77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f81074a0 0x7fa9f81078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:03.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.707+0000 7fa9f77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f81074a0 0x7fa9f81078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48070/0 (socket says 172.21.15.19:48070) 2024-04-25T11:58:03.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.707+0000 7fa9f77fe700 1 -- 172.21.15.19:0/2804837834 learned_addr learned my addr 172.21.15.19:0/2804837834 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:58:03.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.707+0000 7fa9f77fe700 1 -- 172.21.15.19:0/2804837834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f81086a0 con 0x7fa9f81074a0 2024-04-25T11:58:03.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.707+0000 7fa9f77fe700 1 --2- 172.21.15.19:0/2804837834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f81074a0 0x7fa9f81078c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0098a0 tx=0x7fa9ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ffc71280d0b73c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:03.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.707+0000 7fa9f67fc700 1 -- 172.21.15.19:0/2804837834 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ec004160 con 0x7fa9f81074a0 2024-04-25T11:58:03.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.707+0000 7fa9f67fc700 1 -- 172.21.15.19:0/2804837834 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa9ec004300 con 0x7fa9f81074a0 2024-04-25T11:58:03.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.707+0000 7fa9f67fc700 1 -- 172.21.15.19:0/2804837834 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ec011af0 con 0x7fa9f81074a0 2024-04-25T11:58:03.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.708+0000 7fa9fe039700 1 -- 172.21.15.19:0/2804837834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f81074a0 msgr2=0x7fa9f81078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:03.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.708+0000 7fa9fe039700 1 --2- 172.21.15.19:0/2804837834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f81074a0 0x7fa9f81078c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0098a0 tx=0x7fa9ec009bb0 comp rx=0 tx=0).stop 2024-04-25T11:58:03.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.708+0000 7fa9fe039700 1 -- 172.21.15.19:0/2804837834 shutdown_connections 2024-04-25T11:58:03.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.708+0000 7fa9fe039700 1 --2- 172.21.15.19:0/2804837834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f81074a0 0x7fa9f81078c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:03.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.708+0000 7fa9fe039700 1 -- 172.21.15.19:0/2804837834 >> 172.21.15.19:0/2804837834 conn(0x7fa9f8102a40 msgr2=0x7fa9f8104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:03.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.708+0000 7fa9fe039700 1 -- 172.21.15.19:0/2804837834 shutdown_connections 2024-04-25T11:58:03.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.708+0000 7fa9fe039700 1 -- 172.21.15.19:0/2804837834 wait complete. 2024-04-25T11:58:03.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.709+0000 7fa9fe039700 1 Processor -- start 2024-04-25T11:58:03.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.709+0000 7fa9fe039700 1 -- start start 2024-04-25T11:58:03.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.709+0000 7fa9fe039700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f81074a0 0x7fa9f8192600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:03.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.709+0000 7fa9fe039700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f810ba50 con 0x7fa9f81074a0 2024-04-25T11:58:03.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.709+0000 7fa9f77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f81074a0 0x7fa9f8192600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:03.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.709+0000 7fa9f77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f81074a0 0x7fa9f8192600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48072/0 (socket says 172.21.15.19:48072) 2024-04-25T11:58:03.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.709+0000 7fa9f77fe700 1 -- 172.21.15.19:0/4181525577 learned_addr learned my addr 172.21.15.19:0/4181525577 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:58:03.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.709+0000 7fa9f77fe700 1 -- 172.21.15.19:0/4181525577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f8192b40 con 0x7fa9f81074a0 2024-04-25T11:58:03.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.710+0000 7fa9f77fe700 1 --2- 172.21.15.19:0/4181525577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f81074a0 0x7fa9f8192600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec003f90 tx=0x7fa9ec011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:03.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.710+0000 7fa9f4ff9700 1 -- 172.21.15.19:0/4181525577 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ec0039e0 con 0x7fa9f81074a0 2024-04-25T11:58:03.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.710+0000 7fa9f4ff9700 1 -- 172.21.15.19:0/4181525577 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa9ec003b80 con 0x7fa9f81074a0 2024-04-25T11:58:03.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.710+0000 7fa9f4ff9700 1 -- 172.21.15.19:0/4181525577 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9ec0196e0 con 0x7fa9f81074a0 2024-04-25T11:58:03.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.710+0000 7fa9fe039700 1 -- 172.21.15.19:0/4181525577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f8190d80 con 0x7fa9f81074a0 2024-04-25T11:58:03.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.710+0000 7fa9fe039700 1 -- 172.21.15.19:0/4181525577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f8191260 con 0x7fa9f81074a0 2024-04-25T11:58:03.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.711+0000 7fa9f4ff9700 1 -- 172.21.15.19:0/4181525577 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa9ec022020 con 0x7fa9f81074a0 2024-04-25T11:58:03.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.711+0000 7fa9fe039700 1 -- 172.21.15.19:0/4181525577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f80640a0 con 0x7fa9f81074a0 2024-04-25T11:58:03.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.711+0000 7fa9f4ff9700 1 --2- 172.21.15.19:0/4181525577 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa9e00397c0 0x7fa9e003bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:03.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.711+0000 7fa9f4ff9700 1 -- 172.21.15.19:0/4181525577 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa9ec04da10 con 0x7fa9f81074a0 2024-04-25T11:58:03.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.711+0000 7fa9f6ffd700 1 --2- 172.21.15.19:0/4181525577 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa9e00397c0 0x7fa9e003bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:03.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.711+0000 7fa9f6ffd700 1 --2- 172.21.15.19:0/4181525577 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa9e00397c0 0x7fa9e003bc80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006fd0 tx=0x7fa9e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:03.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:02.716+0000 7fa9f4ff9700 1 -- 172.21.15.19:0/4181525577 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9ec01abf0 con 0x7fa9f81074a0 2024-04-25T11:58:03.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:03.042+0000 7fa9fe039700 1 -- 172.21.15.19:0/4181525577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fa9f8196330 con 0x7fa9f81074a0 2024-04-25T11:58:03.627 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:03.044+0000 7fa9f4ff9700 1 -- 172.21.15.19:0/4181525577 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fa9ec01a430 con 0x7fa9f81074a0 2024-04-25T11:58:03.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:03.046+0000 7fa9fe039700 1 -- 172.21.15.19:0/4181525577 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa9e00397c0 msgr2=0x7fa9e003bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:03.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:03.046+0000 7fa9fe039700 1 --2- 172.21.15.19:0/4181525577 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa9e00397c0 0x7fa9e003bc80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006fd0 tx=0x7fa9e8006e40 comp rx=0 tx=0).stop 2024-04-25T11:58:03.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:03.046+0000 7fa9fe039700 1 -- 172.21.15.19:0/4181525577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f81074a0 msgr2=0x7fa9f8192600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:03.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:03.046+0000 7fa9fe039700 1 --2- 172.21.15.19:0/4181525577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f81074a0 0x7fa9f8192600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec003f90 tx=0x7fa9ec011b70 comp rx=0 tx=0).stop 2024-04-25T11:58:03.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:03.046+0000 7fa9fe039700 1 -- 172.21.15.19:0/4181525577 shutdown_connections 2024-04-25T11:58:03.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:03.046+0000 7fa9fe039700 1 --2- 172.21.15.19:0/4181525577 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa9e00397c0 0x7fa9e003bc80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:03.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:03.046+0000 7fa9fe039700 1 --2- 172.21.15.19:0/4181525577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9f81074a0 0x7fa9f8192600 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:03.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:03.046+0000 7fa9fe039700 1 -- 172.21.15.19:0/4181525577 >> 172.21.15.19:0/4181525577 conn(0x7fa9f8102a40 msgr2=0x7fa9f8103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:03.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:03.047+0000 7fa9fe039700 1 -- 172.21.15.19:0/4181525577 shutdown_connections 2024-04-25T11:58:03.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-04-25T11:58:03.047+0000 7fa9fe039700 1 -- 172.21.15.19:0/4181525577 wait complete. 2024-04-25T11:58:03.628 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T11:58:03.628 INFO:teuthology.orchestra.run.smithi019.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T11:58:03.628 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:58:03.628 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:58:03.629 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:58:03.629 INFO:teuthology.orchestra.run.smithi019.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T11:58:03.629 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:58:03.629 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T11:58:03.629 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:58:03.629 INFO:teuthology.orchestra.run.smithi019.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T11:58:03.629 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:58:03.629 INFO:teuthology.orchestra.run.smithi019.stdout: ceph telemetry on 2024-04-25T11:58:03.629 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:58:03.629 INFO:teuthology.orchestra.run.smithi019.stdout:For more information see: 2024-04-25T11:58:03.629 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:58:03.629 INFO:teuthology.orchestra.run.smithi019.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T11:58:03.629 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:58:03.629 INFO:teuthology.orchestra.run.smithi019.stdout:Bootstrap complete. 2024-04-25T11:58:03.664 INFO:tasks.cephadm:Fetching config... 2024-04-25T11:58:03.664 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:58:03.664 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T11:58:03.681 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T11:58:03.681 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:58:03.681 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T11:58:03.737 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T11:58:03.737 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:58:03.737 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T11:58:03.802 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T11:58:03.802 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:58:03.802 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T11:58:03.859 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T11:58:03.859 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6Piaq9f/B27YRiSSrRtQc69+e/F4AyIehtNaz7mPAhfCrjyxWSMlxSEi3q4HX26OJ6ghULMf9eOsk9JwP2Z7vm9UuTEUU0mnoQXuHIpORKqd6azFFQ98wBQQXVGmbwO5pRsHeFziHjDrSJvIoi4wznI4T03ULbjSNJau+e1aBPuOWWzZ8m2jKjeZRnAaK2p46l2ZJO3llMfPVtp/1Ep/QZW1LcB3Az/wbXBDfHNiScld8AwYiLY5ueXDGJB17ZiCt69fD4a86xJJhLk2E7jaHBNNjOgfGDrNfRGM+LzWEMlBqqgg8R2isSTazP45xx/K2AMEl5w78TJkAtl9CdLddbLMDkwJ0XqwIp4XKRqHOeP22luStYJHytIVHEK1e0ZrZJb0+FphAl4mK8id+ued+Z0v930H70wFNMDV9zS9nBzKQ/mPzJxCZfilObQz4kXhDjS0YqG5sycgjzom745EAoSUzkJ0asyyemrIeKADgo0h0a55OsFcqZ3Yh7u8B8qk= ceph-be62e290-02fa-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T11:58:03.932 INFO:teuthology.orchestra.run.smithi019.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6Piaq9f/B27YRiSSrRtQc69+e/F4AyIehtNaz7mPAhfCrjyxWSMlxSEi3q4HX26OJ6ghULMf9eOsk9JwP2Z7vm9UuTEUU0mnoQXuHIpORKqd6azFFQ98wBQQXVGmbwO5pRsHeFziHjDrSJvIoi4wznI4T03ULbjSNJau+e1aBPuOWWzZ8m2jKjeZRnAaK2p46l2ZJO3llMfPVtp/1Ep/QZW1LcB3Az/wbXBDfHNiScld8AwYiLY5ueXDGJB17ZiCt69fD4a86xJJhLk2E7jaHBNNjOgfGDrNfRGM+LzWEMlBqqgg8R2isSTazP45xx/K2AMEl5w78TJkAtl9CdLddbLMDkwJ0XqwIp4XKRqHOeP22luStYJHytIVHEK1e0ZrZJb0+FphAl4mK8id+ued+Z0v930H70wFNMDV9zS9nBzKQ/mPzJxCZfilObQz4kXhDjS0YqG5sycgjzom745EAoSUzkJ0asyyemrIeKADgo0h0a55OsFcqZ3Yh7u8B8qk= ceph-be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:58:03.944 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6Piaq9f/B27YRiSSrRtQc69+e/F4AyIehtNaz7mPAhfCrjyxWSMlxSEi3q4HX26OJ6ghULMf9eOsk9JwP2Z7vm9UuTEUU0mnoQXuHIpORKqd6azFFQ98wBQQXVGmbwO5pRsHeFziHjDrSJvIoi4wznI4T03ULbjSNJau+e1aBPuOWWzZ8m2jKjeZRnAaK2p46l2ZJO3llMfPVtp/1Ep/QZW1LcB3Az/wbXBDfHNiScld8AwYiLY5ueXDGJB17ZiCt69fD4a86xJJhLk2E7jaHBNNjOgfGDrNfRGM+LzWEMlBqqgg8R2isSTazP45xx/K2AMEl5w78TJkAtl9CdLddbLMDkwJ0XqwIp4XKRqHOeP22luStYJHytIVHEK1e0ZrZJb0+FphAl4mK8id+ued+Z0v930H70wFNMDV9zS9nBzKQ/mPzJxCZfilObQz4kXhDjS0YqG5sycgjzom745EAoSUzkJ0asyyemrIeKADgo0h0a55OsFcqZ3Yh7u8B8qk= ceph-be62e290-02fa-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T11:58:03.982 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6Piaq9f/B27YRiSSrRtQc69+e/F4AyIehtNaz7mPAhfCrjyxWSMlxSEi3q4HX26OJ6ghULMf9eOsk9JwP2Z7vm9UuTEUU0mnoQXuHIpORKqd6azFFQ98wBQQXVGmbwO5pRsHeFziHjDrSJvIoi4wznI4T03ULbjSNJau+e1aBPuOWWzZ8m2jKjeZRnAaK2p46l2ZJO3llMfPVtp/1Ep/QZW1LcB3Az/wbXBDfHNiScld8AwYiLY5ueXDGJB17ZiCt69fD4a86xJJhLk2E7jaHBNNjOgfGDrNfRGM+LzWEMlBqqgg8R2isSTazP45xx/K2AMEl5w78TJkAtl9CdLddbLMDkwJ0XqwIp4XKRqHOeP22luStYJHytIVHEK1e0ZrZJb0+FphAl4mK8id+ued+Z0v930H70wFNMDV9zS9nBzKQ/mPzJxCZfilObQz4kXhDjS0YqG5sycgjzom745EAoSUzkJ0asyyemrIeKADgo0h0a55OsFcqZ3Yh7u8B8qk= ceph-be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:58:03.994 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6Piaq9f/B27YRiSSrRtQc69+e/F4AyIehtNaz7mPAhfCrjyxWSMlxSEi3q4HX26OJ6ghULMf9eOsk9JwP2Z7vm9UuTEUU0mnoQXuHIpORKqd6azFFQ98wBQQXVGmbwO5pRsHeFziHjDrSJvIoi4wznI4T03ULbjSNJau+e1aBPuOWWzZ8m2jKjeZRnAaK2p46l2ZJO3llMfPVtp/1Ep/QZW1LcB3Az/wbXBDfHNiScld8AwYiLY5ueXDGJB17ZiCt69fD4a86xJJhLk2E7jaHBNNjOgfGDrNfRGM+LzWEMlBqqgg8R2isSTazP45xx/K2AMEl5w78TJkAtl9CdLddbLMDkwJ0XqwIp4XKRqHOeP22luStYJHytIVHEK1e0ZrZJb0+FphAl4mK8id+ued+Z0v930H70wFNMDV9zS9nBzKQ/mPzJxCZfilObQz4kXhDjS0YqG5sycgjzom745EAoSUzkJ0asyyemrIeKADgo0h0a55OsFcqZ3Yh7u8B8qk= ceph-be62e290-02fa-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T11:58:04.034 INFO:teuthology.orchestra.run.smithi169.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6Piaq9f/B27YRiSSrRtQc69+e/F4AyIehtNaz7mPAhfCrjyxWSMlxSEi3q4HX26OJ6ghULMf9eOsk9JwP2Z7vm9UuTEUU0mnoQXuHIpORKqd6azFFQ98wBQQXVGmbwO5pRsHeFziHjDrSJvIoi4wznI4T03ULbjSNJau+e1aBPuOWWzZ8m2jKjeZRnAaK2p46l2ZJO3llMfPVtp/1Ep/QZW1LcB3Az/wbXBDfHNiScld8AwYiLY5ueXDGJB17ZiCt69fD4a86xJJhLk2E7jaHBNNjOgfGDrNfRGM+LzWEMlBqqgg8R2isSTazP45xx/K2AMEl5w78TJkAtl9CdLddbLMDkwJ0XqwIp4XKRqHOeP22luStYJHytIVHEK1e0ZrZJb0+FphAl4mK8id+ued+Z0v930H70wFNMDV9zS9nBzKQ/mPzJxCZfilObQz4kXhDjS0YqG5sycgjzom745EAoSUzkJ0asyyemrIeKADgo0h0a55OsFcqZ3Yh7u8B8qk= ceph-be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:58:04.045 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T11:58:04.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:04 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/4181525577' entity='client.admin' 2024-04-25T11:58:04.295 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:58:06.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.902+0000 7f31df13a700 1 -- 172.21.15.19:0/4108579864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8102f30 msgr2=0x7f31d8105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:06.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.902+0000 7f31df13a700 1 --2- 172.21.15.19:0/4108579864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8102f30 0x7f31d8105320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009910 tx=0x7f31cc009c20 comp rx=0 tx=0).stop 2024-04-25T11:58:06.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.902+0000 7f31df13a700 1 -- 172.21.15.19:0/4108579864 shutdown_connections 2024-04-25T11:58:06.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.902+0000 7f31df13a700 1 --2- 172.21.15.19:0/4108579864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8102f30 0x7f31d8105320 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:06.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.902+0000 7f31df13a700 1 -- 172.21.15.19:0/4108579864 >> 172.21.15.19:0/4108579864 conn(0x7f31d80fc940 msgr2=0x7f31d80fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:06.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.902+0000 7f31df13a700 1 -- 172.21.15.19:0/4108579864 shutdown_connections 2024-04-25T11:58:06.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.902+0000 7f31df13a700 1 -- 172.21.15.19:0/4108579864 wait complete. 2024-04-25T11:58:06.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.903+0000 7f31df13a700 1 Processor -- start 2024-04-25T11:58:06.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.903+0000 7f31df13a700 1 -- start start 2024-04-25T11:58:06.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.903+0000 7f31df13a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d81a26a0 0x7f31d81a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:06.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.903+0000 7f31df13a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f31d806cb60 con 0x7f31d81a26a0 2024-04-25T11:58:06.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.903+0000 7f31dced6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d81a26a0 0x7f31d81a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:06.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.903+0000 7f31dced6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d81a26a0 0x7f31d81a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48092/0 (socket says 172.21.15.19:48092) 2024-04-25T11:58:06.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.903+0000 7f31dced6700 1 -- 172.21.15.19:0/1219053378 learned_addr learned my addr 172.21.15.19:0/1219053378 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:58:06.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.904+0000 7f31dced6700 1 -- 172.21.15.19:0/1219053378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d81a3000 con 0x7f31d81a26a0 2024-04-25T11:58:06.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.904+0000 7f31dced6700 1 --2- 172.21.15.19:0/1219053378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d81a26a0 0x7f31d81a2ac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f31cc000c00 tx=0x7f31cc009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:06.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.904+0000 7f31d5ffb700 1 -- 172.21.15.19:0/1219053378 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31cc027070 con 0x7f31d81a26a0 2024-04-25T11:58:06.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.904+0000 7f31df13a700 1 -- 172.21.15.19:0/1219053378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d81a32f0 con 0x7f31d81a26a0 2024-04-25T11:58:06.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.904+0000 7f31d5ffb700 1 -- 172.21.15.19:0/1219053378 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f31cc022470 con 0x7f31d81a26a0 2024-04-25T11:58:06.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.904+0000 7f31d5ffb700 1 -- 172.21.15.19:0/1219053378 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31cc00f5d0 con 0x7f31d81a26a0 2024-04-25T11:58:06.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.904+0000 7f31df13a700 1 -- 172.21.15.19:0/1219053378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d81a5e50 con 0x7f31d81a26a0 2024-04-25T11:58:06.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.905+0000 7f31d5ffb700 1 -- 172.21.15.19:0/1219053378 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f31cc02a070 con 0x7f31d81a26a0 2024-04-25T11:58:06.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.905+0000 7f31d5ffb700 1 --2- 172.21.15.19:0/1219053378 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31c8039650 0x7f31c803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.905+0000 7f31d5ffb700 1 -- 172.21.15.19:0/1219053378 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f31cc01e030 con 0x7f31d81a26a0 2024-04-25T11:58:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.906+0000 7f31d7fff700 1 --2- 172.21.15.19:0/1219053378 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31c8039650 0x7f31c803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.906+0000 7f31df13a700 1 -- 172.21.15.19:0/1219053378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31d806d2e0 con 0x7f31d81a26a0 2024-04-25T11:58:06.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.906+0000 7f31d7fff700 1 --2- 172.21.15.19:0/1219053378 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31c8039650 0x7f31c803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f31c4006fd0 tx=0x7f31c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:06.911+0000 7f31d5ffb700 1 -- 172.21.15.19:0/1219053378 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31cc013070 con 0x7f31d81a26a0 2024-04-25T11:58:07.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:07.118+0000 7f31df13a700 1 -- 172.21.15.19:0/1219053378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f31d802e090 con 0x7f31d81a26a0 2024-04-25T11:58:07.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:07.121+0000 7f31d5ffb700 1 -- 172.21.15.19:0/1219053378 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f31cc035030 con 0x7f31d81a26a0 2024-04-25T11:58:07.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:07.124+0000 7f31df13a700 1 -- 172.21.15.19:0/1219053378 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31c8039650 msgr2=0x7f31c803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:07.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:07.124+0000 7f31df13a700 1 --2- 172.21.15.19:0/1219053378 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31c8039650 0x7f31c803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f31c4006fd0 tx=0x7f31c4006e40 comp rx=0 tx=0).stop 2024-04-25T11:58:07.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:07.124+0000 7f31df13a700 1 -- 172.21.15.19:0/1219053378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d81a26a0 msgr2=0x7f31d81a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:07.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:07.124+0000 7f31df13a700 1 --2- 172.21.15.19:0/1219053378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d81a26a0 0x7f31d81a2ac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f31cc000c00 tx=0x7f31cc009e60 comp rx=0 tx=0).stop 2024-04-25T11:58:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:07.124+0000 7f31df13a700 1 -- 172.21.15.19:0/1219053378 shutdown_connections 2024-04-25T11:58:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:07.124+0000 7f31df13a700 1 --2- 172.21.15.19:0/1219053378 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31c8039650 0x7f31c803bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:07.124+0000 7f31df13a700 1 --2- 172.21.15.19:0/1219053378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d81a26a0 0x7f31d81a2ac0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:07.124+0000 7f31df13a700 1 -- 172.21.15.19:0/1219053378 >> 172.21.15.19:0/1219053378 conn(0x7f31d80fc940 msgr2=0x7f31d80fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:07.125+0000 7f31df13a700 1 -- 172.21.15.19:0/1219053378 shutdown_connections 2024-04-25T11:58:07.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:07.125+0000 7f31df13a700 1 -- 172.21.15.19:0/1219053378 wait complete. 2024-04-25T11:58:07.744 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T11:58:07.744 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T11:58:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:58:08.444 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:08 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1219053378' entity='client.admin' 2024-04-25T11:58:09.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:09 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:09.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:09 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:09.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:09 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:58:09.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:09 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:09.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:09 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:58:09.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:09 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:09.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.863+0000 7f1bbea26700 1 -- 172.21.15.19:0/2340391936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb81024c0 msgr2=0x7f1bb81028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:09.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.863+0000 7f1bbea26700 1 --2- 172.21.15.19:0/2340391936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb81024c0 0x7f1bb81028e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009910 tx=0x7f1ba8009c20 comp rx=0 tx=0).stop 2024-04-25T11:58:09.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.863+0000 7f1bbea26700 1 -- 172.21.15.19:0/2340391936 shutdown_connections 2024-04-25T11:58:09.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.863+0000 7f1bbea26700 1 --2- 172.21.15.19:0/2340391936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb81024c0 0x7f1bb81028e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.863+0000 7f1bbea26700 1 -- 172.21.15.19:0/2340391936 >> 172.21.15.19:0/2340391936 conn(0x7f1bb80fda60 msgr2=0x7f1bb80ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.863+0000 7f1bbea26700 1 -- 172.21.15.19:0/2340391936 shutdown_connections 2024-04-25T11:58:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.863+0000 7f1bbea26700 1 -- 172.21.15.19:0/2340391936 wait complete. 2024-04-25T11:58:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.863+0000 7f1bbea26700 1 Processor -- start 2024-04-25T11:58:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.863+0000 7f1bbea26700 1 -- start start 2024-04-25T11:58:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.864+0000 7f1bbea26700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb81024c0 0x7f1bb8076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.864+0000 7f1bbea26700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb8108540 con 0x7f1bb81024c0 2024-04-25T11:58:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.864+0000 7f1bb7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb81024c0 0x7f1bb8076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.864+0000 7f1bb7fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb81024c0 0x7f1bb8076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48112/0 (socket says 172.21.15.19:48112) 2024-04-25T11:58:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.864+0000 7f1bb7fff700 1 -- 172.21.15.19:0/1807142498 learned_addr learned my addr 172.21.15.19:0/1807142498 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:58:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.864+0000 7f1bb7fff700 1 -- 172.21.15.19:0/1807142498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb8076c40 con 0x7f1bb81024c0 2024-04-25T11:58:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.864+0000 7f1bb7fff700 1 --2- 172.21.15.19:0/1807142498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb81024c0 0x7f1bb8076700 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009ec0 tx=0x7f1ba8004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.864+0000 7f1bb5ffb700 1 -- 172.21.15.19:0/1807142498 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ba8027070 con 0x7f1bb81024c0 2024-04-25T11:58:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.864+0000 7f1bb5ffb700 1 -- 172.21.15.19:0/1807142498 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1ba8022470 con 0x7f1bb81024c0 2024-04-25T11:58:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.864+0000 7f1bbea26700 1 -- 172.21.15.19:0/1807142498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb8074e30 con 0x7f1bb81024c0 2024-04-25T11:58:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.864+0000 7f1bb5ffb700 1 -- 172.21.15.19:0/1807142498 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ba800f5f0 con 0x7f1bb81024c0 2024-04-25T11:58:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.864+0000 7f1bbea26700 1 -- 172.21.15.19:0/1807142498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb8075370 con 0x7f1bb81024c0 2024-04-25T11:58:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.865+0000 7f1bb5ffb700 1 -- 172.21.15.19:0/1807142498 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1ba802a070 con 0x7f1bb81024c0 2024-04-25T11:58:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.865+0000 7f1bbea26700 1 -- 172.21.15.19:0/1807142498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb8108cc0 con 0x7f1bb81024c0 2024-04-25T11:58:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.865+0000 7f1bb5ffb700 1 --2- 172.21.15.19:0/1807142498 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f1ba40396b0 0x7f1ba403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.866+0000 7f1bb5ffb700 1 -- 172.21.15.19:0/1807142498 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1ba801e030 con 0x7f1bb81024c0 2024-04-25T11:58:09.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.866+0000 7f1baffff700 1 --2- 172.21.15.19:0/1807142498 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f1ba40396b0 0x7f1ba403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:09.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.866+0000 7f1baffff700 1 --2- 172.21.15.19:0/1807142498 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f1ba40396b0 0x7f1ba403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0006fd0 tx=0x7f1ba0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:09.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:09.871+0000 7f1bb5ffb700 1 -- 172.21.15.19:0/1807142498 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ba8013070 con 0x7f1bb81024c0 2024-04-25T11:58:10.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:10.084+0000 7f1bbea26700 1 -- 172.21.15.19:0/1807142498 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f1bb8190fc0 con 0x7f1ba40396b0 2024-04-25T11:58:10.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:10.087+0000 7f1bb5ffb700 1 -- 172.21.15.19:0/1807142498 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1bb8190fc0 con 0x7f1ba40396b0 2024-04-25T11:58:10.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:10.089+0000 7f1bbea26700 1 -- 172.21.15.19:0/1807142498 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f1ba40396b0 msgr2=0x7f1ba403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:10.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:10.089+0000 7f1bbea26700 1 --2- 172.21.15.19:0/1807142498 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f1ba40396b0 0x7f1ba403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0006fd0 tx=0x7f1ba0006e40 comp rx=0 tx=0).stop 2024-04-25T11:58:10.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:10.089+0000 7f1bbea26700 1 -- 172.21.15.19:0/1807142498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb81024c0 msgr2=0x7f1bb8076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:10.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:10.089+0000 7f1bbea26700 1 --2- 172.21.15.19:0/1807142498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb81024c0 0x7f1bb8076700 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009ec0 tx=0x7f1ba8004690 comp rx=0 tx=0).stop 2024-04-25T11:58:10.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:10.089+0000 7f1bbea26700 1 -- 172.21.15.19:0/1807142498 shutdown_connections 2024-04-25T11:58:10.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:10.089+0000 7f1bbea26700 1 --2- 172.21.15.19:0/1807142498 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f1ba40396b0 0x7f1ba403bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:10.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:10.089+0000 7f1bbea26700 1 --2- 172.21.15.19:0/1807142498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1bb81024c0 0x7f1bb8076700 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:10.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:10.089+0000 7f1bbea26700 1 -- 172.21.15.19:0/1807142498 >> 172.21.15.19:0/1807142498 conn(0x7f1bb80fda60 msgr2=0x7f1bb80fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:10.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:10.089+0000 7f1bbea26700 1 -- 172.21.15.19:0/1807142498 shutdown_connections 2024-04-25T11:58:10.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:10.089+0000 7f1bbea26700 1 -- 172.21.15.19:0/1807142498 wait complete. 2024-04-25T11:58:10.848 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi150 2024-04-25T11:58:10.849 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T11:58:10.849 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.conf 2024-04-25T11:58:10.868 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T11:58:10.868 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:58:10.925 INFO:tasks.cephadm:Adding host smithi150 to orchestrator... 2024-04-25T11:58:10.926 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch host add smithi150 2024-04-25T11:58:11.179 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:58:11.406 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:11 smithi019 ceph-mon[29213]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:58:11.406 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:11 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:11.406 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:11 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:58:11.406 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:11 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:11.406 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:11 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:58:11.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:11 smithi019 ceph-mon[29213]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T11:58:11.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:11 smithi019 ceph-mon[29213]: Updating smithi019:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:11.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:11 smithi019 ceph-mon[29213]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:58:11.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:11 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:11.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:11 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:11.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:11 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:12.260 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:12 smithi019 ceph-mon[29213]: Updating smithi019:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:58:12.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.593+0000 7f56973fa700 1 -- 172.21.15.19:0/1935908596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5690101580 msgr2=0x7f5690101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:12.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.593+0000 7f56973fa700 1 --2- 172.21.15.19:0/1935908596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5690101580 0x7f5690101960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f568c009910 tx=0x7f568c009c20 comp rx=0 tx=0).stop 2024-04-25T11:58:12.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.593+0000 7f56973fa700 1 -- 172.21.15.19:0/1935908596 shutdown_connections 2024-04-25T11:58:12.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.593+0000 7f56973fa700 1 --2- 172.21.15.19:0/1935908596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5690101580 0x7f5690101960 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:12.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.593+0000 7f56973fa700 1 -- 172.21.15.19:0/1935908596 >> 172.21.15.19:0/1935908596 conn(0x7f56900fce50 msgr2=0x7f56900ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.593+0000 7f56973fa700 1 -- 172.21.15.19:0/1935908596 shutdown_connections 2024-04-25T11:58:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.593+0000 7f56973fa700 1 -- 172.21.15.19:0/1935908596 wait complete. 2024-04-25T11:58:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.593+0000 7f56973fa700 1 Processor -- start 2024-04-25T11:58:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.593+0000 7f56973fa700 1 -- start start 2024-04-25T11:58:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.594+0000 7f56973fa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5690101580 0x7f5690197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.594+0000 7f56973fa700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f569010b940 con 0x7f5690101580 2024-04-25T11:58:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.594+0000 7f5695196700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5690101580 0x7f5690197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.594+0000 7f5695196700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5690101580 0x7f5690197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52578/0 (socket says 172.21.15.19:52578) 2024-04-25T11:58:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.594+0000 7f5695196700 1 -- 172.21.15.19:0/1090613991 learned_addr learned my addr 172.21.15.19:0/1090613991 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:58:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.594+0000 7f5695196700 1 -- 172.21.15.19:0/1090613991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5690198440 con 0x7f5690101580 2024-04-25T11:58:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.594+0000 7f5695196700 1 --2- 172.21.15.19:0/1090613991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5690101580 0x7f5690197f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f568c005780 tx=0x7f568c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.594+0000 7f56867fc700 1 -- 172.21.15.19:0/1090613991 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f568c026070 con 0x7f5690101580 2024-04-25T11:58:12.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.594+0000 7f56867fc700 1 -- 172.21.15.19:0/1090613991 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f568c004370 con 0x7f5690101580 2024-04-25T11:58:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.594+0000 7f56867fc700 1 -- 172.21.15.19:0/1090613991 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f568c003a30 con 0x7f5690101580 2024-04-25T11:58:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.594+0000 7f56973fa700 1 -- 172.21.15.19:0/1090613991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5690198730 con 0x7f5690101580 2024-04-25T11:58:12.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.594+0000 7f56973fa700 1 -- 172.21.15.19:0/1090613991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f569019c7d0 con 0x7f5690101580 2024-04-25T11:58:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.595+0000 7f56867fc700 1 -- 172.21.15.19:0/1090613991 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f568c029070 con 0x7f5690101580 2024-04-25T11:58:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.595+0000 7f56973fa700 1 -- 172.21.15.19:0/1090613991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5690102a60 con 0x7f5690101580 2024-04-25T11:58:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.595+0000 7f56867fc700 1 --2- 172.21.15.19:0/1090613991 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f567c039650 0x7f567c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.596+0000 7f56867fc700 1 -- 172.21.15.19:0/1090613991 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f568c01e070 con 0x7f5690101580 2024-04-25T11:58:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.596+0000 7f5694995700 1 --2- 172.21.15.19:0/1090613991 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f567c039650 0x7f567c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:12.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.596+0000 7f5694995700 1 --2- 172.21.15.19:0/1090613991 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f567c039650 0x7f567c03bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5680006fd0 tx=0x7f5680006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:12.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.601+0000 7f56867fc700 1 -- 172.21.15.19:0/1090613991 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f568c013070 con 0x7f5690101580 2024-04-25T11:58:12.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:12.812+0000 7f56973fa700 1 -- 172.21.15.19:0/1090613991 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}) -- 0x7f5690199260 con 0x7f567c039650 2024-04-25T11:58:14.042 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:13 smithi019 ceph-mon[29213]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:58:14.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:13 smithi019 ceph-mon[29213]: Deploying cephadm binary to smithi150 2024-04-25T11:58:16.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:15 smithi019 ceph-mon[29213]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:17.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:17.072+0000 7f56867fc700 1 -- 172.21.15.19:0/1090613991 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f5690199260 con 0x7f567c039650 2024-04-25T11:58:17.066 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi150' with addr '172.21.15.150' 2024-04-25T11:58:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:17.075+0000 7f56973fa700 1 -- 172.21.15.19:0/1090613991 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f567c039650 msgr2=0x7f567c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:17.075+0000 7f56973fa700 1 --2- 172.21.15.19:0/1090613991 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f567c039650 0x7f567c03bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5680006fd0 tx=0x7f5680006e40 comp rx=0 tx=0).stop 2024-04-25T11:58:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:17.075+0000 7f56973fa700 1 -- 172.21.15.19:0/1090613991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5690101580 msgr2=0x7f5690197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:17.075+0000 7f56973fa700 1 --2- 172.21.15.19:0/1090613991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5690101580 0x7f5690197f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f568c005780 tx=0x7f568c0057b0 comp rx=0 tx=0).stop 2024-04-25T11:58:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:17.075+0000 7f56973fa700 1 -- 172.21.15.19:0/1090613991 shutdown_connections 2024-04-25T11:58:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:17.075+0000 7f56973fa700 1 --2- 172.21.15.19:0/1090613991 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f567c039650 0x7f567c03bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:17.075+0000 7f56973fa700 1 --2- 172.21.15.19:0/1090613991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5690101580 0x7f5690197f00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:17.075+0000 7f56973fa700 1 -- 172.21.15.19:0/1090613991 >> 172.21.15.19:0/1090613991 conn(0x7f56900fce50 msgr2=0x7f569006a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:17.075+0000 7f56973fa700 1 -- 172.21.15.19:0/1090613991 shutdown_connections 2024-04-25T11:58:17.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:17.075+0000 7f56973fa700 1 -- 172.21.15.19:0/1090613991 wait complete. 2024-04-25T11:58:17.677 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T11:58:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:58:18.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:18 smithi019 ceph-mon[29213]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:18.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:18.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:18 smithi019 ceph-mon[29213]: Added host smithi150 2024-04-25T11:58:18.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:58:18.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:19.258 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:19 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.493+0000 7f8075ae4700 1 -- 172.21.15.19:0/4105754908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8070103650 msgr2=0x7f8070103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.494+0000 7f8075ae4700 1 --2- 172.21.15.19:0/4105754908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8070103650 0x7f8070103a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8060009910 tx=0x7f8060009c20 comp rx=0 tx=0).stop 2024-04-25T11:58:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.494+0000 7f8075ae4700 1 -- 172.21.15.19:0/4105754908 shutdown_connections 2024-04-25T11:58:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.494+0000 7f8075ae4700 1 --2- 172.21.15.19:0/4105754908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8070103650 0x7f8070103a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.494+0000 7f8075ae4700 1 -- 172.21.15.19:0/4105754908 >> 172.21.15.19:0/4105754908 conn(0x7f80700fcea0 msgr2=0x7f80700ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.494+0000 7f8075ae4700 1 -- 172.21.15.19:0/4105754908 shutdown_connections 2024-04-25T11:58:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.494+0000 7f8075ae4700 1 -- 172.21.15.19:0/4105754908 wait complete. 2024-04-25T11:58:19.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.495+0000 7f8075ae4700 1 Processor -- start 2024-04-25T11:58:19.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.495+0000 7f8075ae4700 1 -- start start 2024-04-25T11:58:19.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.495+0000 7f8075ae4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8070103650 0x7f8070197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:19.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.495+0000 7f8075ae4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f807010cb10 con 0x7f8070103650 2024-04-25T11:58:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.495+0000 7f806f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8070103650 0x7f8070197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.495+0000 7f806f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8070103650 0x7f8070197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52606/0 (socket says 172.21.15.19:52606) 2024-04-25T11:58:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.495+0000 7f806f7fe700 1 -- 172.21.15.19:0/2246085664 learned_addr learned my addr 172.21.15.19:0/2246085664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:58:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.495+0000 7f806f7fe700 1 -- 172.21.15.19:0/2246085664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8070198490 con 0x7f8070103650 2024-04-25T11:58:19.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.495+0000 7f806f7fe700 1 --2- 172.21.15.19:0/2246085664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8070103650 0x7f8070197f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f80600057a0 tx=0x7f8060005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.495+0000 7f806d7fa700 1 -- 172.21.15.19:0/2246085664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8060026070 con 0x7f8070103650 2024-04-25T11:58:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.495+0000 7f806d7fa700 1 -- 172.21.15.19:0/2246085664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8060004370 con 0x7f8070103650 2024-04-25T11:58:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.496+0000 7f8075ae4700 1 -- 172.21.15.19:0/2246085664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8070198720 con 0x7f8070103650 2024-04-25T11:58:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.496+0000 7f806d7fa700 1 -- 172.21.15.19:0/2246085664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8060003b90 con 0x7f8070103650 2024-04-25T11:58:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.496+0000 7f8075ae4700 1 -- 172.21.15.19:0/2246085664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8070198b80 con 0x7f8070103650 2024-04-25T11:58:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.497+0000 7f8075ae4700 1 -- 172.21.15.19:0/2246085664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f807006b190 con 0x7f8070103650 2024-04-25T11:58:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.497+0000 7f806d7fa700 1 -- 172.21.15.19:0/2246085664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8060029070 con 0x7f8070103650 2024-04-25T11:58:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.497+0000 7f806d7fa700 1 --2- 172.21.15.19:0/2246085664 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f805c039650 0x7f805c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:19.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.497+0000 7f806d7fa700 1 -- 172.21.15.19:0/2246085664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f806001e070 con 0x7f8070103650 2024-04-25T11:58:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.497+0000 7f8067fff700 1 --2- 172.21.15.19:0/2246085664 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f805c039650 0x7f805c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:19.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.497+0000 7f8067fff700 1 --2- 172.21.15.19:0/2246085664 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f805c039650 0x7f805c03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8058006fd0 tx=0x7f8058006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:19.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.502+0000 7f806d7fa700 1 -- 172.21.15.19:0/2246085664 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8060013070 con 0x7f8070103650 2024-04-25T11:58:19.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.715+0000 7f8075ae4700 1 -- 172.21.15.19:0/2246085664 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80701992b0 con 0x7f805c039650 2024-04-25T11:58:19.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.716+0000 7f806d7fa700 1 -- 172.21.15.19:0/2246085664 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f80701992b0 con 0x7f805c039650 2024-04-25T11:58:19.710 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:58:19.710 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}] 2024-04-25T11:58:19.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.718+0000 7f8075ae4700 1 -- 172.21.15.19:0/2246085664 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f805c039650 msgr2=0x7f805c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.718+0000 7f8075ae4700 1 --2- 172.21.15.19:0/2246085664 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f805c039650 0x7f805c03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8058006fd0 tx=0x7f8058006e40 comp rx=0 tx=0).stop 2024-04-25T11:58:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.718+0000 7f8075ae4700 1 -- 172.21.15.19:0/2246085664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8070103650 msgr2=0x7f8070197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.718+0000 7f8075ae4700 1 --2- 172.21.15.19:0/2246085664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8070103650 0x7f8070197f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f80600057a0 tx=0x7f8060005dc0 comp rx=0 tx=0).stop 2024-04-25T11:58:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.718+0000 7f8075ae4700 1 -- 172.21.15.19:0/2246085664 shutdown_connections 2024-04-25T11:58:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.718+0000 7f8075ae4700 1 --2- 172.21.15.19:0/2246085664 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f805c039650 0x7f805c03bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.718+0000 7f8075ae4700 1 --2- 172.21.15.19:0/2246085664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8070103650 0x7f8070197f50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:19.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.718+0000 7f8075ae4700 1 -- 172.21.15.19:0/2246085664 >> 172.21.15.19:0/2246085664 conn(0x7f80700fcea0 msgr2=0x7f80701043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.718+0000 7f8075ae4700 1 -- 172.21.15.19:0/2246085664 shutdown_connections 2024-04-25T11:58:19.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:19.718+0000 7f8075ae4700 1 -- 172.21.15.19:0/2246085664 wait complete. 2024-04-25T11:58:20.327 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi169 2024-04-25T11:58:20.327 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:58:20.327 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.conf 2024-04-25T11:58:20.346 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:58:20.346 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:58:20.404 INFO:tasks.cephadm:Adding host smithi169 to orchestrator... 2024-04-25T11:58:20.404 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch host add smithi169 2024-04-25T11:58:20.653 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:58:20.937 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:20 smithi019 ceph-mon[29213]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:20.937 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:20 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:21.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:21 smithi019 ceph-mon[29213]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:58:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.184+0000 7fefccca4700 1 -- 172.21.15.19:0/389431170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc81036b0 msgr2=0x7fefc8103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.184+0000 7fefccca4700 1 --2- 172.21.15.19:0/389431170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc81036b0 0x7fefc8103a90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009910 tx=0x7fefb8009c20 comp rx=0 tx=0).stop 2024-04-25T11:58:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.184+0000 7fefccca4700 1 -- 172.21.15.19:0/389431170 shutdown_connections 2024-04-25T11:58:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.184+0000 7fefccca4700 1 --2- 172.21.15.19:0/389431170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc81036b0 0x7fefc8103a90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.184+0000 7fefccca4700 1 -- 172.21.15.19:0/389431170 >> 172.21.15.19:0/389431170 conn(0x7fefc80fce40 msgr2=0x7fefc80ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.184+0000 7fefccca4700 1 -- 172.21.15.19:0/389431170 shutdown_connections 2024-04-25T11:58:22.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.184+0000 7fefccca4700 1 -- 172.21.15.19:0/389431170 wait complete. 2024-04-25T11:58:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.184+0000 7fefccca4700 1 Processor -- start 2024-04-25T11:58:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.184+0000 7fefccca4700 1 -- start start 2024-04-25T11:58:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.185+0000 7fefccca4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc81036b0 0x7fefc8101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.185+0000 7fefccca4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fefc810caf0 con 0x7fefc81036b0 2024-04-25T11:58:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.185+0000 7fefc659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc81036b0 0x7fefc8101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.185+0000 7fefc659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc81036b0 0x7fefc8101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57280/0 (socket says 172.21.15.19:57280) 2024-04-25T11:58:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.185+0000 7fefc659c700 1 -- 172.21.15.19:0/3989529781 learned_addr learned my addr 172.21.15.19:0/3989529781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:58:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.185+0000 7fefc659c700 1 -- 172.21.15.19:0/3989529781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc8103200 con 0x7fefc81036b0 2024-04-25T11:58:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.185+0000 7fefc659c700 1 --2- 172.21.15.19:0/3989529781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc81036b0 0x7fefc8101310 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fefb80057a0 tx=0x7fefb8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.185+0000 7fefb77fe700 1 -- 172.21.15.19:0/3989529781 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefb8026070 con 0x7fefc81036b0 2024-04-25T11:58:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.185+0000 7fefb77fe700 1 -- 172.21.15.19:0/3989529781 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fefb8004370 con 0x7fefc81036b0 2024-04-25T11:58:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.185+0000 7fefccca4700 1 -- 172.21.15.19:0/3989529781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc8101850 con 0x7fefc81036b0 2024-04-25T11:58:22.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.185+0000 7fefb77fe700 1 -- 172.21.15.19:0/3989529781 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefb8003b90 con 0x7fefc81036b0 2024-04-25T11:58:22.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.185+0000 7fefccca4700 1 -- 172.21.15.19:0/3989529781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc8101d10 con 0x7fefc81036b0 2024-04-25T11:58:22.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.186+0000 7fefb77fe700 1 -- 172.21.15.19:0/3989529781 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fefb8029070 con 0x7fefc81036b0 2024-04-25T11:58:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.186+0000 7fefccca4700 1 -- 172.21.15.19:0/3989529781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc806b0f0 con 0x7fefc81036b0 2024-04-25T11:58:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.187+0000 7fefb77fe700 1 --2- 172.21.15.19:0/3989529781 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fefb00396b0 0x7fefb003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.187+0000 7fefb77fe700 1 -- 172.21.15.19:0/3989529781 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fefb801e070 con 0x7fefc81036b0 2024-04-25T11:58:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.187+0000 7fefc5d9b700 1 --2- 172.21.15.19:0/3989529781 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fefb00396b0 0x7fefb003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.187+0000 7fefc5d9b700 1 --2- 172.21.15.19:0/3989529781 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fefb00396b0 0x7fefb003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fefbc006fd0 tx=0x7fefbc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.192+0000 7fefb77fe700 1 -- 172.21.15.19:0/3989529781 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefb8013070 con 0x7fefc81036b0 2024-04-25T11:58:22.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:22.402+0000 7fefccca4700 1 -- 172.21.15.19:0/3989529781 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}) -- 0x7fefc8102840 con 0x7fefb00396b0 2024-04-25T11:58:23.042 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:22 smithi019 ceph-mon[29213]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:23.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:23 smithi019 ceph-mon[29213]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:58:23.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:23.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:23.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:23.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:23.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:58:23.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:23.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:58:25.042 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:24 smithi019 ceph-mon[29213]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:25.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:24 smithi019 ceph-mon[29213]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T11:58:25.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:24 smithi019 ceph-mon[29213]: Deploying cephadm binary to smithi169 2024-04-25T11:58:25.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:24 smithi019 ceph-mon[29213]: Updating smithi150:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:25.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:24 smithi019 ceph-mon[29213]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:58:25.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:24 smithi019 ceph-mon[29213]: Updating smithi150:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:58:26.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:26.713+0000 7fefb77fe700 1 -- 172.21.15.19:0/3989529781 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fefc8102840 con 0x7fefb00396b0 2024-04-25T11:58:26.706 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi169' with addr '172.21.15.169' 2024-04-25T11:58:26.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:26.715+0000 7fefccca4700 1 -- 172.21.15.19:0/3989529781 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fefb00396b0 msgr2=0x7fefb003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:26.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:26.715+0000 7fefccca4700 1 --2- 172.21.15.19:0/3989529781 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fefb00396b0 0x7fefb003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fefbc006fd0 tx=0x7fefbc006e40 comp rx=0 tx=0).stop 2024-04-25T11:58:26.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:26.715+0000 7fefccca4700 1 -- 172.21.15.19:0/3989529781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc81036b0 msgr2=0x7fefc8101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:26.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:26.715+0000 7fefccca4700 1 --2- 172.21.15.19:0/3989529781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc81036b0 0x7fefc8101310 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fefb80057a0 tx=0x7fefb8005dc0 comp rx=0 tx=0).stop 2024-04-25T11:58:26.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:26.715+0000 7fefccca4700 1 -- 172.21.15.19:0/3989529781 shutdown_connections 2024-04-25T11:58:26.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:26.715+0000 7fefccca4700 1 --2- 172.21.15.19:0/3989529781 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fefb00396b0 0x7fefb003bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:26.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:26.715+0000 7fefccca4700 1 --2- 172.21.15.19:0/3989529781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc81036b0 0x7fefc8101310 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:26.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:26.715+0000 7fefccca4700 1 -- 172.21.15.19:0/3989529781 >> 172.21.15.19:0/3989529781 conn(0x7fefc80fce40 msgr2=0x7fefc8104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:26.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:26.716+0000 7fefccca4700 1 -- 172.21.15.19:0/3989529781 shutdown_connections 2024-04-25T11:58:26.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:26.716+0000 7fefccca4700 1 -- 172.21.15.19:0/3989529781 wait complete. 2024-04-25T11:58:27.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:26 smithi019 ceph-mon[29213]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:27.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:27.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:27.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:27.301 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T11:58:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:58:27.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:27 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:27.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:27 smithi019 ceph-mon[29213]: Added host smithi169 2024-04-25T11:58:27.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:27 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:58:27.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:27 smithi019 ceph-mon[29213]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:27.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:27 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:28.919 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:28 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:29.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.146+0000 7f9033aa8700 1 -- 172.21.15.19:0/1470026892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f902c1015f0 msgr2=0x7f902c1019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:29.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.146+0000 7f9033aa8700 1 --2- 172.21.15.19:0/1470026892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f902c1015f0 0x7f902c1019d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9028009910 tx=0x7f9028009c20 comp rx=0 tx=0).stop 2024-04-25T11:58:29.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.146+0000 7f9033aa8700 1 -- 172.21.15.19:0/1470026892 shutdown_connections 2024-04-25T11:58:29.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.146+0000 7f9033aa8700 1 --2- 172.21.15.19:0/1470026892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f902c1015f0 0x7f902c1019d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:29.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.146+0000 7f9033aa8700 1 -- 172.21.15.19:0/1470026892 >> 172.21.15.19:0/1470026892 conn(0x7f902c076ea0 msgr2=0x7f902c0772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:29.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.146+0000 7f9033aa8700 1 -- 172.21.15.19:0/1470026892 shutdown_connections 2024-04-25T11:58:29.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.146+0000 7f9033aa8700 1 -- 172.21.15.19:0/1470026892 wait complete. 2024-04-25T11:58:29.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.147+0000 7f9033aa8700 1 Processor -- start 2024-04-25T11:58:29.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.147+0000 7f9033aa8700 1 -- start start 2024-04-25T11:58:29.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.147+0000 7f9033aa8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f902c1015f0 0x7f902c19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:29.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.147+0000 7f9033aa8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f902c10a860 con 0x7f902c1015f0 2024-04-25T11:58:29.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.147+0000 7f9031844700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f902c1015f0 0x7f902c19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:29.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.147+0000 7f9031844700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f902c1015f0 0x7f902c19a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57294/0 (socket says 172.21.15.19:57294) 2024-04-25T11:58:29.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.147+0000 7f9031844700 1 -- 172.21.15.19:0/3981032406 learned_addr learned my addr 172.21.15.19:0/3981032406 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:58:29.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.147+0000 7f9031844700 1 -- 172.21.15.19:0/3981032406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902c19a660 con 0x7f902c1015f0 2024-04-25T11:58:29.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.147+0000 7f9031844700 1 --2- 172.21.15.19:0/3981032406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f902c1015f0 0x7f902c19a120 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9028005780 tx=0x7f90280057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.148+0000 7f9022ffd700 1 -- 172.21.15.19:0/3981032406 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9028026070 con 0x7f902c1015f0 2024-04-25T11:58:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.148+0000 7f9022ffd700 1 -- 172.21.15.19:0/3981032406 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9028004370 con 0x7f902c1015f0 2024-04-25T11:58:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.148+0000 7f9033aa8700 1 -- 172.21.15.19:0/3981032406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f902c19a950 con 0x7f902c1015f0 2024-04-25T11:58:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.148+0000 7f9022ffd700 1 -- 172.21.15.19:0/3981032406 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9028003a30 con 0x7f902c1015f0 2024-04-25T11:58:29.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.148+0000 7f9033aa8700 1 -- 172.21.15.19:0/3981032406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f902c19e9a0 con 0x7f902c1015f0 2024-04-25T11:58:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.149+0000 7f9022ffd700 1 -- 172.21.15.19:0/3981032406 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9028029070 con 0x7f902c1015f0 2024-04-25T11:58:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.149+0000 7f9033aa8700 1 -- 172.21.15.19:0/3981032406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f902c102a80 con 0x7f902c1015f0 2024-04-25T11:58:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.149+0000 7f9022ffd700 1 --2- 172.21.15.19:0/3981032406 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9018039650 0x7f901803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.149+0000 7f9022ffd700 1 -- 172.21.15.19:0/3981032406 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f902801e070 con 0x7f902c1015f0 2024-04-25T11:58:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.149+0000 7f9031043700 1 --2- 172.21.15.19:0/3981032406 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9018039650 0x7f901803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:29.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.149+0000 7f9031043700 1 --2- 172.21.15.19:0/3981032406 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9018039650 0x7f901803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f901c006fd0 tx=0x7f901c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:29.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.154+0000 7f9022ffd700 1 -- 172.21.15.19:0/3981032406 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9028013070 con 0x7f902c1015f0 2024-04-25T11:58:29.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.366+0000 7f9033aa8700 1 -- 172.21.15.19:0/3981032406 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f902c19b430 con 0x7f9018039650 2024-04-25T11:58:29.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.367+0000 7f9022ffd700 1 -- 172.21.15.19:0/3981032406 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f902c19b430 con 0x7f9018039650 2024-04-25T11:58:29.360 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T11:58:29.361 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}, {"addr": "172.21.15.169", "hostname": "smithi169", "labels": [], "status": ""}] 2024-04-25T11:58:29.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.369+0000 7f9033aa8700 1 -- 172.21.15.19:0/3981032406 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9018039650 msgr2=0x7f901803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:29.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.369+0000 7f9033aa8700 1 --2- 172.21.15.19:0/3981032406 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9018039650 0x7f901803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f901c006fd0 tx=0x7f901c006e40 comp rx=0 tx=0).stop 2024-04-25T11:58:29.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.369+0000 7f9033aa8700 1 -- 172.21.15.19:0/3981032406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f902c1015f0 msgr2=0x7f902c19a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:29.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.370+0000 7f9033aa8700 1 --2- 172.21.15.19:0/3981032406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f902c1015f0 0x7f902c19a120 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9028005780 tx=0x7f90280057b0 comp rx=0 tx=0).stop 2024-04-25T11:58:29.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.370+0000 7f9033aa8700 1 -- 172.21.15.19:0/3981032406 shutdown_connections 2024-04-25T11:58:29.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.370+0000 7f9033aa8700 1 --2- 172.21.15.19:0/3981032406 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9018039650 0x7f901803bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:29.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.370+0000 7f9033aa8700 1 --2- 172.21.15.19:0/3981032406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f902c1015f0 0x7f902c19a120 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:29.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.370+0000 7f9033aa8700 1 -- 172.21.15.19:0/3981032406 >> 172.21.15.19:0/3981032406 conn(0x7f902c076ea0 msgr2=0x7f902c102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:29.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.370+0000 7f9033aa8700 1 -- 172.21.15.19:0/3981032406 shutdown_connections 2024-04-25T11:58:29.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:29.370+0000 7f9033aa8700 1 -- 172.21.15.19:0/3981032406 wait complete. 2024-04-25T11:58:29.993 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T11:58:29.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T11:58:30.236 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:58:30.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:30 smithi019 ceph-mon[29213]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:30.574 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:30 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:30.574 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:30 smithi019 ceph-mon[29213]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:58:31.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.668+0000 7f636f583700 1 -- 172.21.15.19:0/552253929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6368108550 msgr2=0x7f6368108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:31.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.668+0000 7f636f583700 1 --2- 172.21.15.19:0/552253929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6368108550 0x7f6368108930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6364009910 tx=0x7f6364009c20 comp rx=0 tx=0).stop 2024-04-25T11:58:31.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.669+0000 7f636f583700 1 -- 172.21.15.19:0/552253929 shutdown_connections 2024-04-25T11:58:31.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.669+0000 7f636f583700 1 --2- 172.21.15.19:0/552253929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6368108550 0x7f6368108930 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:31.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.669+0000 7f636f583700 1 -- 172.21.15.19:0/552253929 >> 172.21.15.19:0/552253929 conn(0x7f63680fdf70 msgr2=0x7f6368100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.669+0000 7f636f583700 1 -- 172.21.15.19:0/552253929 shutdown_connections 2024-04-25T11:58:31.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.669+0000 7f636f583700 1 -- 172.21.15.19:0/552253929 wait complete. 2024-04-25T11:58:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.669+0000 7f636f583700 1 Processor -- start 2024-04-25T11:58:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.670+0000 7f636f583700 1 -- start start 2024-04-25T11:58:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.670+0000 7f636f583700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6368108550 0x7f6368197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.670+0000 7f636f583700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f636810cb00 con 0x7f6368108550 2024-04-25T11:58:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.670+0000 7f636d31f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6368108550 0x7f6368197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:31.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.670+0000 7f636d31f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6368108550 0x7f6368197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54588/0 (socket says 172.21.15.19:54588) 2024-04-25T11:58:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.670+0000 7f636d31f700 1 -- 172.21.15.19:0/2809855885 learned_addr learned my addr 172.21.15.19:0/2809855885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:58:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.670+0000 7f636d31f700 1 -- 172.21.15.19:0/2809855885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6368198460 con 0x7f6368108550 2024-04-25T11:58:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.670+0000 7f636d31f700 1 --2- 172.21.15.19:0/2809855885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6368108550 0x7f6368197f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6364005780 tx=0x7f63640057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.670+0000 7f635e7fc700 1 -- 172.21.15.19:0/2809855885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6364026070 con 0x7f6368108550 2024-04-25T11:58:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.670+0000 7f635e7fc700 1 -- 172.21.15.19:0/2809855885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6364004370 con 0x7f6368108550 2024-04-25T11:58:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.670+0000 7f636f583700 1 -- 172.21.15.19:0/2809855885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63681986f0 con 0x7f6368108550 2024-04-25T11:58:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.671+0000 7f635e7fc700 1 -- 172.21.15.19:0/2809855885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6364003af0 con 0x7f6368108550 2024-04-25T11:58:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.671+0000 7f636f583700 1 -- 172.21.15.19:0/2809855885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6368198b50 con 0x7f6368108550 2024-04-25T11:58:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.671+0000 7f635e7fc700 1 -- 172.21.15.19:0/2809855885 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6364029070 con 0x7f6368108550 2024-04-25T11:58:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.671+0000 7f636f583700 1 -- 172.21.15.19:0/2809855885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6368103bc0 con 0x7f6368108550 2024-04-25T11:58:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.672+0000 7f635e7fc700 1 --2- 172.21.15.19:0/2809855885 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f63540396b0 0x7f635403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:31.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.672+0000 7f635e7fc700 1 -- 172.21.15.19:0/2809855885 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f636401e070 con 0x7f6368108550 2024-04-25T11:58:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.672+0000 7f636cb1e700 1 --2- 172.21.15.19:0/2809855885 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f63540396b0 0x7f635403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:31.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.672+0000 7f636cb1e700 1 --2- 172.21.15.19:0/2809855885 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f63540396b0 0x7f635403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6358006fd0 tx=0x7f6358006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:31.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.677+0000 7f635e7fc700 1 -- 172.21.15.19:0/2809855885 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6364013070 con 0x7f6368108550 2024-04-25T11:58:31.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:31.890+0000 7f636f583700 1 -- 172.21.15.19:0/2809855885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f6368199280 con 0x7f6368108550 2024-04-25T11:58:32.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:32.330+0000 7f635e7fc700 1 -- 172.21.15.19:0/2809855885 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f636400d340 con 0x7f6368108550 2024-04-25T11:58:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:32.332+0000 7f636f583700 1 -- 172.21.15.19:0/2809855885 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f63540396b0 msgr2=0x7f635403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:32.332+0000 7f636f583700 1 --2- 172.21.15.19:0/2809855885 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f63540396b0 0x7f635403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6358006fd0 tx=0x7f6358006e40 comp rx=0 tx=0).stop 2024-04-25T11:58:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:32.332+0000 7f636f583700 1 -- 172.21.15.19:0/2809855885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6368108550 msgr2=0x7f6368197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:32.332+0000 7f636f583700 1 --2- 172.21.15.19:0/2809855885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6368108550 0x7f6368197f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6364005780 tx=0x7f63640057b0 comp rx=0 tx=0).stop 2024-04-25T11:58:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:32.333+0000 7f636f583700 1 -- 172.21.15.19:0/2809855885 shutdown_connections 2024-04-25T11:58:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:32.333+0000 7f636f583700 1 --2- 172.21.15.19:0/2809855885 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f63540396b0 0x7f635403bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:32.333+0000 7f636f583700 1 --2- 172.21.15.19:0/2809855885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6368108550 0x7f6368197f20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:32.333+0000 7f636f583700 1 -- 172.21.15.19:0/2809855885 >> 172.21.15.19:0/2809855885 conn(0x7f63680fdf70 msgr2=0x7f63680ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:32.333+0000 7f636f583700 1 -- 172.21.15.19:0/2809855885 shutdown_connections 2024-04-25T11:58:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:32.333+0000 7f636f583700 1 -- 172.21.15.19:0/2809855885 wait complete. 2024-04-25T11:58:32.327 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-04-25T11:58:32.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:32 smithi019 ceph-mon[29213]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:32.624 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:32 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2809855885' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T11:58:32.906 INFO:tasks.cephadm:Adding mon.a on smithi019 2024-04-25T11:58:32.906 INFO:tasks.cephadm:Adding mon.b on smithi150 2024-04-25T11:58:32.906 INFO:tasks.cephadm:Adding mon.c on smithi169 2024-04-25T11:58:32.907 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi019:172.21.15.19=a;smithi150:172.21.15.150=b;smithi169:172.21.15.169=c' 2024-04-25T11:58:33.141 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:33.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:33 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2809855885' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T11:58:33.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:33 smithi019 ceph-mon[29213]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:58:33.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:33.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:33.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:33.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:33.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:58:33.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:33.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:58:34.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.712+0000 7f5d91d89700 1 -- 172.21.15.169:0/311546253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8c103610 msgr2=0x7f5d8c1039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:34.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.712+0000 7f5d91d89700 1 --2- 172.21.15.169:0/311546253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8c103610 0x7f5d8c1039f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009910 tx=0x7f5d7c009c20 comp rx=0 tx=0).stop 2024-04-25T11:58:34.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.712+0000 7f5d91d89700 1 -- 172.21.15.169:0/311546253 shutdown_connections 2024-04-25T11:58:34.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.712+0000 7f5d91d89700 1 --2- 172.21.15.169:0/311546253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8c103610 0x7f5d8c1039f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:34.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.712+0000 7f5d91d89700 1 -- 172.21.15.169:0/311546253 >> 172.21.15.169:0/311546253 conn(0x7f5d8c0fce60 msgr2=0x7f5d8c0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:34.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.712+0000 7f5d91d89700 1 -- 172.21.15.169:0/311546253 shutdown_connections 2024-04-25T11:58:34.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.712+0000 7f5d91d89700 1 -- 172.21.15.169:0/311546253 wait complete. 2024-04-25T11:58:34.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.713+0000 7f5d91d89700 1 Processor -- start 2024-04-25T11:58:34.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.713+0000 7f5d91d89700 1 -- start start 2024-04-25T11:58:34.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.713+0000 7f5d91d89700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8c103610 0x7f5d8c101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:34.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.713+0000 7f5d91d89700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c10caf0 con 0x7f5d8c103610 2024-04-25T11:58:34.709 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.714+0000 7f5d8b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8c103610 0x7f5d8c101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:34.709 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.714+0000 7f5d8b7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8c103610 0x7f5d8c101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.169:51894/0 (socket says 172.21.15.169:51894) 2024-04-25T11:58:34.709 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.714+0000 7f5d8b7fe700 1 -- 172.21.15.169:0/3409093970 learned_addr learned my addr 172.21.15.169:0/3409093970 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:58:34.709 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.714+0000 7f5d8b7fe700 1 -- 172.21.15.169:0/3409093970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8c103160 con 0x7f5d8c103610 2024-04-25T11:58:34.709 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.714+0000 7f5d8b7fe700 1 --2- 172.21.15.169:0/3409093970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8c103610 0x7f5d8c101270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0057a0 tx=0x7f5d7c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:34.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.714+0000 7f5d88ff9700 1 -- 172.21.15.169:0/3409093970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d7c026070 con 0x7f5d8c103610 2024-04-25T11:58:34.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.714+0000 7f5d88ff9700 1 -- 172.21.15.169:0/3409093970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5d7c004370 con 0x7f5d8c103610 2024-04-25T11:58:34.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.714+0000 7f5d91d89700 1 -- 172.21.15.169:0/3409093970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8c1017b0 con 0x7f5d8c103610 2024-04-25T11:58:34.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.714+0000 7f5d88ff9700 1 -- 172.21.15.169:0/3409093970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d7c003b90 con 0x7f5d8c103610 2024-04-25T11:58:34.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.714+0000 7f5d91d89700 1 -- 172.21.15.169:0/3409093970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d8c101c70 con 0x7f5d8c103610 2024-04-25T11:58:34.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.715+0000 7f5d88ff9700 1 -- 172.21.15.169:0/3409093970 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5d7c029070 con 0x7f5d8c103610 2024-04-25T11:58:34.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.715+0000 7f5d91d89700 1 -- 172.21.15.169:0/3409093970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d8c06b1d0 con 0x7f5d8c103610 2024-04-25T11:58:34.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.715+0000 7f5d88ff9700 1 --2- 172.21.15.169:0/3409093970 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5d74039650 0x7f5d7403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:34.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.715+0000 7f5d88ff9700 1 -- 172.21.15.169:0/3409093970 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5d7c01e070 con 0x7f5d8c103610 2024-04-25T11:58:34.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.716+0000 7f5d8affd700 1 --2- 172.21.15.169:0/3409093970 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5d74039650 0x7f5d7403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:34.711 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.716+0000 7f5d8affd700 1 --2- 172.21.15.169:0/3409093970 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5d74039650 0x7f5d7403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5d80006fd0 tx=0x7f5d80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:34.714 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.721+0000 7f5d88ff9700 1 -- 172.21.15.169:0/3409093970 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d7c013070 con 0x7f5d8c103610 2024-04-25T11:58:34.924 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.930+0000 7f5d91d89700 1 -- 172.21.15.169:0/3409093970 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi150:172.21.15.150=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}) -- 0x7f5d8c102860 con 0x7f5d74039650 2024-04-25T11:58:34.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.938+0000 7f5d88ff9700 1 -- 172.21.15.169:0/3409093970 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5d8c102860 con 0x7f5d74039650 2024-04-25T11:58:34.931 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mon update... 2024-04-25T11:58:34.933 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.939+0000 7f5d91d89700 1 -- 172.21.15.169:0/3409093970 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5d74039650 msgr2=0x7f5d7403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:34.933 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.939+0000 7f5d91d89700 1 --2- 172.21.15.169:0/3409093970 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5d74039650 0x7f5d7403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5d80006fd0 tx=0x7f5d80006e40 comp rx=0 tx=0).stop 2024-04-25T11:58:34.933 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.939+0000 7f5d91d89700 1 -- 172.21.15.169:0/3409093970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8c103610 msgr2=0x7f5d8c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:34.934 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.939+0000 7f5d91d89700 1 --2- 172.21.15.169:0/3409093970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8c103610 0x7f5d8c101270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0057a0 tx=0x7f5d7c005dc0 comp rx=0 tx=0).stop 2024-04-25T11:58:34.934 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.940+0000 7f5d91d89700 1 -- 172.21.15.169:0/3409093970 shutdown_connections 2024-04-25T11:58:34.934 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.940+0000 7f5d91d89700 1 --2- 172.21.15.169:0/3409093970 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5d74039650 0x7f5d7403bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:34.934 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.940+0000 7f5d91d89700 1 --2- 172.21.15.169:0/3409093970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d8c103610 0x7f5d8c101270 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:34.934 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.940+0000 7f5d91d89700 1 -- 172.21.15.169:0/3409093970 >> 172.21.15.169:0/3409093970 conn(0x7f5d8c0fce60 msgr2=0x7f5d8c1043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:34.934 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.940+0000 7f5d91d89700 1 -- 172.21.15.169:0/3409093970 shutdown_connections 2024-04-25T11:58:34.934 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:34.940+0000 7f5d91d89700 1 -- 172.21.15.169:0/3409093970 wait complete. 2024-04-25T11:58:35.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:34 smithi019 ceph-mon[29213]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T11:58:35.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:34 smithi019 ceph-mon[29213]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:35.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:34 smithi019 ceph-mon[29213]: Updating smithi169:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:35.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:34 smithi019 ceph-mon[29213]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:58:35.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:34 smithi019 ceph-mon[29213]: Updating smithi169:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:58:35.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:34 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:35.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:34 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:35.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:34 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:35.464 DEBUG:teuthology.orchestra.run.smithi150:mon.b> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T11:58:35.466 DEBUG:teuthology.orchestra.run.smithi169:mon.c> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T11:58:35.468 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T11:58:35.469 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T11:58:36.167 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:36.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:35 smithi019 ceph-mon[29213]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:36.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:35 smithi019 ceph-mon[29213]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi150:172.21.15.150=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:58:36.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:35 smithi019 ceph-mon[29213]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi150:172.21.15.150=b;smithi169:172.21.15.169=c;count:3 2024-04-25T11:58:36.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:36.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:58:36.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:36.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:58:36.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:36.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:58:36.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:36.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:35 smithi019 ceph-mon[29213]: Deploying daemon mon.c on smithi169 2024-04-25T11:58:38.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:37 smithi019 ceph-mon[29213]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:38.336 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.342+0000 7f14a137c700 1 -- 172.21.15.169:0/4107839601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c1084e0 msgr2=0x7f149c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:38.336 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.342+0000 7f14a137c700 1 --2- 172.21.15.169:0/4107839601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c1084e0 0x7f149c1088c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f148c009910 tx=0x7f148c009c20 comp rx=0 tx=0).stop 2024-04-25T11:58:38.336 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.342+0000 7f14a137c700 1 -- 172.21.15.169:0/4107839601 shutdown_connections 2024-04-25T11:58:38.336 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.342+0000 7f14a137c700 1 --2- 172.21.15.169:0/4107839601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c1084e0 0x7f149c1088c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:38.336 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.342+0000 7f14a137c700 1 -- 172.21.15.169:0/4107839601 >> 172.21.15.169:0/4107839601 conn(0x7f149c0fdf80 msgr2=0x7f149c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:38.336 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.342+0000 7f14a137c700 1 -- 172.21.15.169:0/4107839601 shutdown_connections 2024-04-25T11:58:38.336 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.342+0000 7f14a137c700 1 -- 172.21.15.169:0/4107839601 wait complete. 2024-04-25T11:58:38.337 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.342+0000 7f14a137c700 1 Processor -- start 2024-04-25T11:58:38.337 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.343+0000 7f14a137c700 1 -- start start 2024-04-25T11:58:38.338 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.343+0000 7f14a137c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c1084e0 0x7f149c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:38.338 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.343+0000 7f14a137c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f149c10ca80 con 0x7f149c1084e0 2024-04-25T11:58:38.338 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.343+0000 7f149affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c1084e0 0x7f149c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:38.338 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.343+0000 7f149affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c1084e0 0x7f149c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.169:51910/0 (socket says 172.21.15.169:51910) 2024-04-25T11:58:38.338 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.343+0000 7f149affd700 1 -- 172.21.15.169:0/465360677 learned_addr learned my addr 172.21.15.169:0/465360677 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:58:38.338 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.343+0000 7f149affd700 1 -- 172.21.15.169:0/465360677 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149c1984a0 con 0x7f149c1084e0 2024-04-25T11:58:38.338 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.344+0000 7f149affd700 1 --2- 172.21.15.169:0/465360677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c1084e0 0x7f149c197f60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f148c005780 tx=0x7f148c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:38.338 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.344+0000 7f1483fff700 1 -- 172.21.15.169:0/465360677 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f148c026070 con 0x7f149c1084e0 2024-04-25T11:58:38.339 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.344+0000 7f1483fff700 1 -- 172.21.15.169:0/465360677 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f148c004370 con 0x7f149c1084e0 2024-04-25T11:58:38.339 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.344+0000 7f14a137c700 1 -- 172.21.15.169:0/465360677 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149c198790 con 0x7f149c1084e0 2024-04-25T11:58:38.339 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.344+0000 7f1483fff700 1 -- 172.21.15.169:0/465360677 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f148c003a30 con 0x7f149c1084e0 2024-04-25T11:58:38.339 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.344+0000 7f14a137c700 1 -- 172.21.15.169:0/465360677 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f149c19c7e0 con 0x7f149c1084e0 2024-04-25T11:58:38.340 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.345+0000 7f14a137c700 1 -- 172.21.15.169:0/465360677 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f149c103b90 con 0x7f149c1084e0 2024-04-25T11:58:38.340 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.345+0000 7f1483fff700 1 -- 172.21.15.169:0/465360677 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f148c029070 con 0x7f149c1084e0 2024-04-25T11:58:38.340 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.345+0000 7f1483fff700 1 --2- 172.21.15.169:0/465360677 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f14840396b0 0x7f148403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:38.340 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.345+0000 7f1483fff700 1 -- 172.21.15.169:0/465360677 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f148c01e070 con 0x7f149c1084e0 2024-04-25T11:58:38.340 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.345+0000 7f149a7fc700 1 --2- 172.21.15.169:0/465360677 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f14840396b0 0x7f148403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:38.340 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.346+0000 7f149a7fc700 1 --2- 172.21.15.169:0/465360677 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f14840396b0 0x7f148403bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1490006fd0 tx=0x7f1490006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:38.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.351+0000 7f1483fff700 1 -- 172.21.15.169:0/465360677 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f148c013070 con 0x7f149c1084e0 2024-04-25T11:58:38.633 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.639+0000 7f14a137c700 1 -- 172.21.15.169:0/465360677 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f149c199270 con 0x7f149c1084e0 2024-04-25T11:58:38.633 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.640+0000 7f1483fff700 1 -- 172.21.15.169:0/465360677 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f148c030090 con 0x7f149c1084e0 2024-04-25T11:58:38.634 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:58:38.634 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","modified":"2024-04-25T11:56:56.493140Z","created":"2024-04-25T11:56:56.493140Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T11:58:38.636 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.642+0000 7f14a137c700 1 -- 172.21.15.169:0/465360677 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f14840396b0 msgr2=0x7f148403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:38.636 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.642+0000 7f14a137c700 1 --2- 172.21.15.169:0/465360677 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f14840396b0 0x7f148403bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1490006fd0 tx=0x7f1490006e40 comp rx=0 tx=0).stop 2024-04-25T11:58:38.636 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.642+0000 7f14a137c700 1 -- 172.21.15.169:0/465360677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c1084e0 msgr2=0x7f149c197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:38.636 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.642+0000 7f14a137c700 1 --2- 172.21.15.169:0/465360677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c1084e0 0x7f149c197f60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f148c005780 tx=0x7f148c0057b0 comp rx=0 tx=0).stop 2024-04-25T11:58:38.636 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.642+0000 7f14a137c700 1 -- 172.21.15.169:0/465360677 shutdown_connections 2024-04-25T11:58:38.636 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.642+0000 7f14a137c700 1 --2- 172.21.15.169:0/465360677 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f14840396b0 0x7f148403bb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:38.637 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.642+0000 7f14a137c700 1 --2- 172.21.15.169:0/465360677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f149c1084e0 0x7f149c197f60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:38.637 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.642+0000 7f14a137c700 1 -- 172.21.15.169:0/465360677 >> 172.21.15.169:0/465360677 conn(0x7f149c0fdf80 msgr2=0x7f149c109f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:38.637 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.642+0000 7f14a137c700 1 -- 172.21.15.169:0/465360677 shutdown_connections 2024-04-25T11:58:38.637 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:38.642+0000 7f14a137c700 1 -- 172.21.15.169:0/465360677 wait complete. 2024-04-25T11:58:38.637 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-04-25T11:58:39.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:38 smithi019 ceph-mon[29213]: from='client.? 172.21.15.169:0/465360677' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:58:40.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:39 smithi019 ceph-mon[29213]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:40.720 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T11:58:40.720 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T11:58:40.720 INFO:journalctl@ceph.mon.c.smithi169.stdout: e1 2024-04-25T11:58:40.720 INFO:journalctl@ceph.mon.c.smithi169.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T11:58:40.720 INFO:journalctl@ceph.mon.c.smithi169.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T11:58:40.720 INFO:journalctl@ceph.mon.c.smithi169.stdout: legacy client fscid: -1 2024-04-25T11:58:40.720 INFO:journalctl@ceph.mon.c.smithi169.stdout: 2024-04-25T11:58:40.720 INFO:journalctl@ceph.mon.c.smithi169.stdout: No filesystems configured 2024-04-25T11:58:40.720 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T11:58:40.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T11:58:40.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T11:58:40.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T11:58:40.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T11:58:40.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T11:58:40.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T11:58:40.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T11:58:40.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T11:58:40.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T11:58:40.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:40.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:40.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:58:40.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:40.721 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:58:40.722 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:40.722 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:40.722 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:40.722 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:40.922 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T11:58:40.922 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T11:58:41.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:58:41.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:41.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:58:41.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:41.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T11:58:41.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: Deploying cephadm binary to smithi169 2024-04-25T11:58:41.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: Updating smithi150:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:41.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:58:41.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: Updating smithi150:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:58:41.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:41.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: Added host smithi169 2024-04-25T11:58:41.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:58:41.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:41.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:41.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:58:41.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:41.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2809855885' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T11:58:41.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2809855885' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T11:58:41.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:58:41.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:58:41.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:41.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:58:41.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T11:58:41.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:41.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: Updating smithi169:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:41.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:58:41.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: Updating smithi169:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:58:41.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:41.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi150:172.21.15.150=b;smithi169:172.21.15.169=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:58:41.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi150:172.21.15.150=b;smithi169:172.21.15.169=c;count:3 2024-04-25T11:58:41.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:58:41.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:41.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:58:41.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:41.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:58:41.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:41.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: Deploying daemon mon.c on smithi169 2024-04-25T11:58:41.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:41.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: from='client.? 172.21.15.169:0/465360677' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:58:41.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:41.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 ceph-mon[32710]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T11:58:41.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 bash[32691]: e2e3dded6e416cf7ae1d646a70c1c618d3df20954e5f7e5202ce919f62a81848 2024-04-25T11:58:41.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:40 smithi169 systemd[1]: Started Ceph mon.c for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T11:58:41.157 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:58:45.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.880+0000 7f123eff1700 1 -- 172.21.15.169:0/4007813121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12280035c0 msgr2=0x7f1228005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:45.876 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.880+0000 7f123eff1700 1 --2- 172.21.15.169:0/4007813121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12280035c0 0x7f1228005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1238074d10 tx=0x7f122c013e90 comp rx=0 tx=0).stop 2024-04-25T11:58:45.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.880+0000 7f123eff1700 1 -- 172.21.15.169:0/4007813121 shutdown_connections 2024-04-25T11:58:45.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.880+0000 7f123eff1700 1 --2- 172.21.15.169:0/4007813121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12280035c0 0x7f1228005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:45.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.880+0000 7f123eff1700 1 -- 172.21.15.169:0/4007813121 >> 172.21.15.169:0/4007813121 conn(0x7f12380fe050 msgr2=0x7f1238100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:45.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.880+0000 7f123eff1700 1 -- 172.21.15.169:0/4007813121 shutdown_connections 2024-04-25T11:58:45.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.880+0000 7f123eff1700 1 -- 172.21.15.169:0/4007813121 wait complete. 2024-04-25T11:58:45.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.881+0000 7f123eff1700 1 Processor -- start 2024-04-25T11:58:45.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.881+0000 7f123eff1700 1 -- start start 2024-04-25T11:58:45.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.881+0000 7f123eff1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12280035c0 0x7f12381a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:45.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.881+0000 7f123eff1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381a49a0 0x7f123819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:45.877 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.881+0000 7f123eff1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1238110be0 con 0x7f12280035c0 2024-04-25T11:58:45.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.881+0000 7f123eff1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1238110a60 con 0x7f12381a49a0 2024-04-25T11:58:45.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.881+0000 7f1237fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381a49a0 0x7f123819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:45.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.881+0000 7f1237fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381a49a0 0x7f123819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:38904/0 (socket says 172.21.15.169:38904) 2024-04-25T11:58:45.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.881+0000 7f1237fff700 1 -- 172.21.15.169:0/1220340667 learned_addr learned my addr 172.21.15.169:0/1220340667 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:58:45.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.881+0000 7f123cd8d700 1 --2- 172.21.15.169:0/1220340667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12280035c0 0x7f12381a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:45.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.881+0000 7f1237fff700 1 -- 172.21.15.169:0/1220340667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381a49a0 msgr2=0x7f123819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T11:58:45.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.881+0000 7f1237fff700 1 -- 172.21.15.169:0/1220340667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381a49a0 msgr2=0x7f123819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T11:58:45.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.881+0000 7f1237fff700 1 --2- 172.21.15.169:0/1220340667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381a49a0 0x7f123819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:58:45.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.881+0000 7f1237fff700 1 --2- 172.21.15.169:0/1220340667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381a49a0 0x7f123819e620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:58:45.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.882+0000 7f123cd8d700 1 -- 172.21.15.169:0/1220340667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381a49a0 msgr2=0x7f123819e620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:58:45.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.882+0000 7f123cd8d700 1 --2- 172.21.15.169:0/1220340667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381a49a0 0x7f123819e620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:45.878 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.882+0000 7f123cd8d700 1 -- 172.21.15.169:0/1220340667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12381a4fe0 con 0x7f12280035c0 2024-04-25T11:58:45.879 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.882+0000 7f123cd8d700 1 --2- 172.21.15.169:0/1220340667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12280035c0 0x7f12381a4460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f122c01c000 tx=0x7f122c00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:45.879 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.882+0000 7f1235ffb700 1 -- 172.21.15.169:0/1220340667 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f122c021040 con 0x7f12280035c0 2024-04-25T11:58:45.879 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.882+0000 7f123eff1700 1 -- 172.21.15.169:0/1220340667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123819eb60 con 0x7f12280035c0 2024-04-25T11:58:45.879 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.882+0000 7f1235ffb700 1 -- 172.21.15.169:0/1220340667 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f122c004b80 con 0x7f12280035c0 2024-04-25T11:58:45.879 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.882+0000 7f1235ffb700 1 -- 172.21.15.169:0/1220340667 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f122c026d10 con 0x7f12280035c0 2024-04-25T11:58:45.879 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.883+0000 7f123eff1700 1 -- 172.21.15.169:0/1220340667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f123819f0f0 con 0x7f12280035c0 2024-04-25T11:58:45.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.883+0000 7f1235ffb700 1 -- 172.21.15.169:0/1220340667 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f122c02d020 con 0x7f12280035c0 2024-04-25T11:58:45.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.883+0000 7f123eff1700 1 -- 172.21.15.169:0/1220340667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1238076520 con 0x7f12280035c0 2024-04-25T11:58:45.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.884+0000 7f1235ffb700 1 --2- 172.21.15.169:0/1220340667 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f121c0393f0 0x7f121c03b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:45.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.884+0000 7f1235ffb700 1 -- 172.21.15.169:0/1220340667 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f122c019070 con 0x7f12280035c0 2024-04-25T11:58:45.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.884+0000 7f1237fff700 1 --2- 172.21.15.169:0/1220340667 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f121c0393f0 0x7f121c03b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:45.880 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.884+0000 7f1237fff700 1 --2- 172.21.15.169:0/1220340667 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f121c0393f0 0x7f121c03b8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1224009180 tx=0x7f1224008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:45.883 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:45.889+0000 7f1235ffb700 1 -- 172.21.15.169:0/1220340667 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f122c034020 con 0x7f12280035c0 2024-04-25T11:58:46.151 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:58:46.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: mon.a calling monitor election 2024-04-25T11:58:46.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:46.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:46.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:46.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:46.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: mon.c calling monitor election 2024-04-25T11:58:46.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:46.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:46.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:46.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:46.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:46.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: monmap epoch 2 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: fsid be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: last_changed 2024-04-25T11:58:40.731434+0000 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: created 2024-04-25T11:56:56.493140+0000 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: min_mon_release 19 (squid) 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: election_strategy: 1 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: fsmap 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: mgrmap e13: x(active, since 50s) 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: overall HEALTH_OK 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:46.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:46.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:58:46.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:46.165 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:46.172+0000 7f123eff1700 1 -- 172.21.15.169:0/1220340667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f12380008d0 con 0x7f12280035c0 2024-04-25T11:58:46.166 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:46.172+0000 7f1235ffb700 1 -- 172.21.15.169:0/1220340667 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f122c025370 con 0x7f12280035c0 2024-04-25T11:58:46.167 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:58:46.167 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":2,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","modified":"2024-04-25T11:58:40.731434Z","created":"2024-04-25T11:56:56.493140Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T11:58:46.169 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:46.175+0000 7f123eff1700 1 -- 172.21.15.169:0/1220340667 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f121c0393f0 msgr2=0x7f121c03b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:46.169 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:46.175+0000 7f123eff1700 1 --2- 172.21.15.169:0/1220340667 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f121c0393f0 0x7f121c03b8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1224009180 tx=0x7f1224008040 comp rx=0 tx=0).stop 2024-04-25T11:58:46.169 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:46.175+0000 7f123eff1700 1 -- 172.21.15.169:0/1220340667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12280035c0 msgr2=0x7f12381a4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:46.169 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:46.175+0000 7f123eff1700 1 --2- 172.21.15.169:0/1220340667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12280035c0 0x7f12381a4460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f122c01c000 tx=0x7f122c00fc60 comp rx=0 tx=0).stop 2024-04-25T11:58:46.169 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:46.175+0000 7f123eff1700 1 -- 172.21.15.169:0/1220340667 shutdown_connections 2024-04-25T11:58:46.169 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:46.175+0000 7f123eff1700 1 --2- 172.21.15.169:0/1220340667 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f121c0393f0 0x7f121c03b8b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:46.170 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:46.175+0000 7f123eff1700 1 --2- 172.21.15.169:0/1220340667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12280035c0 0x7f12381a4460 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:46.170 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:46.175+0000 7f123eff1700 1 --2- 172.21.15.169:0/1220340667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12381a49a0 0x7f123819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:46.170 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:46.175+0000 7f123eff1700 1 -- 172.21.15.169:0/1220340667 >> 172.21.15.169:0/1220340667 conn(0x7f12380fe050 msgr2=0x7f12381042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:46.170 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:46.175+0000 7f123eff1700 1 -- 172.21.15.169:0/1220340667 shutdown_connections 2024-04-25T11:58:46.170 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:46.175+0000 7f123eff1700 1 -- 172.21.15.169:0/1220340667 wait complete. 2024-04-25T11:58:46.170 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 2 2024-04-25T11:58:46.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:58:46.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: mon.a calling monitor election 2024-04-25T11:58:46.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:46.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:46.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:46.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:46.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: mon.c calling monitor election 2024-04-25T11:58:46.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:46.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:46.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:46.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:46.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:46.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T11:58:46.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: monmap epoch 2 2024-04-25T11:58:46.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: fsid be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:58:46.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: last_changed 2024-04-25T11:58:40.731434+0000 2024-04-25T11:58:46.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: created 2024-04-25T11:56:56.493140+0000 2024-04-25T11:58:46.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: min_mon_release 19 (squid) 2024-04-25T11:58:46.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: election_strategy: 1 2024-04-25T11:58:46.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T11:58:46.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T11:58:46.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: fsmap 2024-04-25T11:58:46.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:58:46.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: mgrmap e13: x(active, since 50s) 2024-04-25T11:58:46.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: overall HEALTH_OK 2024-04-25T11:58:46.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:46.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:46.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:46.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:58:46.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:47.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:46 smithi019 ceph-mon[29213]: Deploying daemon mon.b on smithi150 2024-04-25T11:58:47.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:46 smithi019 ceph-mon[29213]: from='client.? 172.21.15.169:0/1220340667' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:58:47.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:46 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:47.151 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:46 smithi169 ceph-mon[32710]: Deploying daemon mon.b on smithi150 2024-04-25T11:58:47.151 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:46 smithi169 ceph-mon[32710]: from='client.? 172.21.15.169:0/1220340667' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:58:47.151 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:46 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:47.752 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T11:58:47.753 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T11:58:47.999 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:58:48.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:47 smithi019 ceph-mon[29213]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:48.121 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:47 smithi169 ceph-mon[32710]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:49.526 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.530+0000 7f903b0bf700 1 -- 172.21.15.169:0/2862400759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9034103710 msgr2=0x7f9034103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:49.526 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.530+0000 7f903b0bf700 1 --2- 172.21.15.169:0/2862400759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9034103710 0x7f9034103af0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9028009910 tx=0x7f9028009c20 comp rx=0 tx=0).stop 2024-04-25T11:58:49.527 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.530+0000 7f903b0bf700 1 -- 172.21.15.169:0/2862400759 shutdown_connections 2024-04-25T11:58:49.527 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.530+0000 7f903b0bf700 1 --2- 172.21.15.169:0/2862400759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9034103710 0x7f9034103af0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:49.527 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.530+0000 7f903b0bf700 1 -- 172.21.15.169:0/2862400759 >> 172.21.15.169:0/2862400759 conn(0x7f90340fcf40 msgr2=0x7f90340ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:49.527 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.530+0000 7f903b0bf700 1 -- 172.21.15.169:0/2862400759 shutdown_connections 2024-04-25T11:58:49.527 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.530+0000 7f903b0bf700 1 -- 172.21.15.169:0/2862400759 wait complete. 2024-04-25T11:58:49.527 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.531+0000 7f903b0bf700 1 Processor -- start 2024-04-25T11:58:49.527 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.531+0000 7f903b0bf700 1 -- start start 2024-04-25T11:58:49.527 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.531+0000 7f903b0bf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9034103710 0x7f903419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:49.527 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.531+0000 7f903b0bf700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f903419aa00 0x7f903419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:49.527 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.531+0000 7f903b0bf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f903410cbf0 con 0x7f9034103710 2024-04-25T11:58:49.527 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.531+0000 7f903b0bf700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f903410ca70 con 0x7f903419aa00 2024-04-25T11:58:49.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.531+0000 7f9038e5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9034103710 0x7f903419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:49.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.531+0000 7f9038e5b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9034103710 0x7f903419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.169:59258/0 (socket says 172.21.15.169:59258) 2024-04-25T11:58:49.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.531+0000 7f9038e5b700 1 -- 172.21.15.169:0/953147972 learned_addr learned my addr 172.21.15.169:0/953147972 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:58:49.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.531+0000 7f9033fff700 1 --2- 172.21.15.169:0/953147972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f903419aa00 0x7f903419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:49.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.532+0000 7f9033fff700 1 -- 172.21.15.169:0/953147972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f903419aa00 msgr2=0x7f903419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T11:58:49.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.532+0000 7f9033fff700 1 -- 172.21.15.169:0/953147972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f903419aa00 msgr2=0x7f903419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T11:58:49.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.532+0000 7f9033fff700 1 --2- 172.21.15.169:0/953147972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f903419aa00 0x7f903419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:58:49.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.532+0000 7f9038e5b700 1 -- 172.21.15.169:0/953147972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f903419aa00 msgr2=0x7f903419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:49.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.532+0000 7f9033fff700 1 --2- 172.21.15.169:0/953147972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f903419aa00 0x7f903419ed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:58:49.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.532+0000 7f9038e5b700 1 --2- 172.21.15.169:0/953147972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f903419aa00 0x7f903419ed90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:49.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.532+0000 7f9038e5b700 1 -- 172.21.15.169:0/953147972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903419b040 con 0x7f9034103710 2024-04-25T11:58:49.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.532+0000 7f9038e5b700 1 --2- 172.21.15.169:0/953147972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9034103710 0x7f903419a4c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9028006010 tx=0x7f9028005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:49.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.532+0000 7f9031ffb700 1 -- 172.21.15.169:0/953147972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9028005dc0 con 0x7f9034103710 2024-04-25T11:58:49.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.532+0000 7f903b0bf700 1 -- 172.21.15.169:0/953147972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903419f2d0 con 0x7f9034103710 2024-04-25T11:58:49.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.532+0000 7f9031ffb700 1 -- 172.21.15.169:0/953147972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f902800be30 con 0x7f9034103710 2024-04-25T11:58:49.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.532+0000 7f9031ffb700 1 -- 172.21.15.169:0/953147972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f902800f460 con 0x7f9034103710 2024-04-25T11:58:49.529 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.532+0000 7f903b0bf700 1 -- 172.21.15.169:0/953147972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903419f7d0 con 0x7f9034103710 2024-04-25T11:58:49.530 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.533+0000 7f903b0bf700 1 -- 172.21.15.169:0/953147972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f903406b530 con 0x7f9034103710 2024-04-25T11:58:49.530 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.533+0000 7f9031ffb700 1 -- 172.21.15.169:0/953147972 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f902802a070 con 0x7f9034103710 2024-04-25T11:58:49.530 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.533+0000 7f9031ffb700 1 --2- 172.21.15.169:0/953147972 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f90240393f0 0x7f902403b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:49.530 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.534+0000 7f9031ffb700 1 -- 172.21.15.169:0/953147972 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f902801e030 con 0x7f9034103710 2024-04-25T11:58:49.530 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.534+0000 7f9033fff700 1 --2- 172.21.15.169:0/953147972 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f90240393f0 0x7f902403b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:49.530 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.534+0000 7f9033fff700 1 --2- 172.21.15.169:0/953147972 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f90240393f0 0x7f902403b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9020009850 tx=0x7f9020008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:49.532 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.539+0000 7f9031ffb700 1 -- 172.21.15.169:0/953147972 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9028013070 con 0x7f9034103710 2024-04-25T11:58:49.809 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.815+0000 7f903b0bf700 1 -- 172.21.15.169:0/953147972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f903419fa80 con 0x7f9034103710 2024-04-25T11:58:49.810 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.816+0000 7f9031ffb700 1 -- 172.21.15.169:0/953147972 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f9028031090 con 0x7f9034103710 2024-04-25T11:58:49.810 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:58:49.810 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":2,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","modified":"2024-04-25T11:58:40.731434Z","created":"2024-04-25T11:56:56.493140Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T11:58:49.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.818+0000 7f903b0bf700 1 -- 172.21.15.169:0/953147972 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f90240393f0 msgr2=0x7f902403b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:49.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.818+0000 7f903b0bf700 1 --2- 172.21.15.169:0/953147972 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f90240393f0 0x7f902403b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9020009850 tx=0x7f9020008040 comp rx=0 tx=0).stop 2024-04-25T11:58:49.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.818+0000 7f903b0bf700 1 -- 172.21.15.169:0/953147972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9034103710 msgr2=0x7f903419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:49.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.818+0000 7f903b0bf700 1 --2- 172.21.15.169:0/953147972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9034103710 0x7f903419a4c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9028006010 tx=0x7f9028005030 comp rx=0 tx=0).stop 2024-04-25T11:58:49.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.818+0000 7f903b0bf700 1 -- 172.21.15.169:0/953147972 shutdown_connections 2024-04-25T11:58:49.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.818+0000 7f903b0bf700 1 --2- 172.21.15.169:0/953147972 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f90240393f0 0x7f902403b8b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:49.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.818+0000 7f903b0bf700 1 --2- 172.21.15.169:0/953147972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9034103710 0x7f903419a4c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:49.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.818+0000 7f903b0bf700 1 --2- 172.21.15.169:0/953147972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f903419aa00 0x7f903419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:49.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.818+0000 7f903b0bf700 1 -- 172.21.15.169:0/953147972 >> 172.21.15.169:0/953147972 conn(0x7f90340fcf40 msgr2=0x7f90341088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:49.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.818+0000 7f903b0bf700 1 -- 172.21.15.169:0/953147972 shutdown_connections 2024-04-25T11:58:49.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:49.818+0000 7f903b0bf700 1 -- 172.21.15.169:0/953147972 wait complete. 2024-04-25T11:58:49.813 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 2 2024-04-25T11:58:50.151 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:49 smithi169 ceph-mon[32710]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:50.151 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:49 smithi169 ceph-mon[32710]: from='client.? 172.21.15.169:0/953147972' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:58:50.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:49 smithi019 ceph-mon[29213]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:50.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:49 smithi019 ceph-mon[29213]: from='client.? 172.21.15.169:0/953147972' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:58:50.312 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:50 smithi150 systemd[1]: Started Ceph mon.b for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T11:58:51.405 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T11:58:51.405 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T11:58:51.649 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:58:55.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:58:55.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: mon.a calling monitor election 2024-04-25T11:58:55.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: mon.c calling monitor election 2024-04-25T11:58:55.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: mon.b calling monitor election 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: monmap epoch 3 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: fsid be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: last_changed 2024-04-25T11:58:50.062784+0000 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: created 2024-04-25T11:56:56.493140+0000 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: min_mon_release 19 (squid) 2024-04-25T11:58:55.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: election_strategy: 1 2024-04-25T11:58:55.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T11:58:55.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T11:58:55.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-25T11:58:55.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: fsmap 2024-04-25T11:58:55.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:58:55.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: mgrmap e13: x(active, since 60s) 2024-04-25T11:58:55.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: overall HEALTH_OK 2024-04-25T11:58:55.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:55.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:55.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:55.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:55.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:58:55.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: mon.a calling monitor election 2024-04-25T11:58:55.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: mon.c calling monitor election 2024-04-25T11:58:55.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:55.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:55.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: mon.b calling monitor election 2024-04-25T11:58:55.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:55.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T11:58:55.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: monmap epoch 3 2024-04-25T11:58:55.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: fsid be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:58:55.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: last_changed 2024-04-25T11:58:50.062784+0000 2024-04-25T11:58:55.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: created 2024-04-25T11:56:56.493140+0000 2024-04-25T11:58:55.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: min_mon_release 19 (squid) 2024-04-25T11:58:55.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: election_strategy: 1 2024-04-25T11:58:55.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T11:58:55.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T11:58:55.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-25T11:58:55.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: fsmap 2024-04-25T11:58:55.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:58:55.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: mgrmap e13: x(active, since 60s) 2024-04-25T11:58:55.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: overall HEALTH_OK 2024-04-25T11:58:55.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:55.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:55.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:55.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:58:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: mon.a calling monitor election 2024-04-25T11:58:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: mon.c calling monitor election 2024-04-25T11:58:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:58:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: mon.b calling monitor election 2024-04-25T11:58:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:55.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T11:58:55.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: monmap epoch 3 2024-04-25T11:58:55.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: fsid be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:58:55.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: last_changed 2024-04-25T11:58:50.062784+0000 2024-04-25T11:58:55.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: created 2024-04-25T11:56:56.493140+0000 2024-04-25T11:58:55.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: min_mon_release 19 (squid) 2024-04-25T11:58:55.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: election_strategy: 1 2024-04-25T11:58:55.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-04-25T11:58:55.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: 1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.c 2024-04-25T11:58:55.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-25T11:58:55.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: fsmap 2024-04-25T11:58:55.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:58:55.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: mgrmap e13: x(active, since 60s) 2024-04-25T11:58:55.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: overall HEALTH_OK 2024-04-25T11:58:55.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:55.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:55.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:55.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:56.070 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.074+0000 7ff720748700 1 -- 172.21.15.169:0/726154397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7040035c0 msgr2=0x7ff704005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:56.070 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.074+0000 7ff720748700 1 --2- 172.21.15.169:0/726154397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7040035c0 0x7ff704005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff718101380 tx=0x7ff71400feb0 comp rx=0 tx=0).stop 2024-04-25T11:58:56.070 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.074+0000 7ff720748700 1 -- 172.21.15.169:0/726154397 shutdown_connections 2024-04-25T11:58:56.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.074+0000 7ff720748700 1 --2- 172.21.15.169:0/726154397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7040035c0 0x7ff704005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:56.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.074+0000 7ff720748700 1 -- 172.21.15.169:0/726154397 >> 172.21.15.169:0/726154397 conn(0x7ff7180fcf80 msgr2=0x7ff7180ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:56.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.074+0000 7ff720748700 1 -- 172.21.15.169:0/726154397 shutdown_connections 2024-04-25T11:58:56.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.074+0000 7ff720748700 1 -- 172.21.15.169:0/726154397 wait complete. 2024-04-25T11:58:56.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.075+0000 7ff720748700 1 Processor -- start 2024-04-25T11:58:56.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.075+0000 7ff720748700 1 -- start start 2024-04-25T11:58:56.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.075+0000 7ff720748700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7040035c0 0x7ff7181030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:56.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.075+0000 7ff720748700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7181035f0 0x7ff718103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:56.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.075+0000 7ff720748700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7181a4400 0x7ff7181a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:56.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.075+0000 7ff720748700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff718109490 con 0x7ff7040035c0 2024-04-25T11:58:56.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.075+0000 7ff720748700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff718109310 con 0x7ff7181035f0 2024-04-25T11:58:56.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.075+0000 7ff720748700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff718109610 con 0x7ff7181a4400 2024-04-25T11:58:56.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.075+0000 7ff71ece5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7181a4400 0x7ff7181a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:56.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.075+0000 7ff71ece5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7181a4400 0x7ff7181a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54956/0 (socket says 172.21.15.169:54956) 2024-04-25T11:58:56.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.075+0000 7ff71ece5700 1 -- 172.21.15.169:0/211345678 learned_addr learned my addr 172.21.15.169:0/211345678 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:58:56.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.075+0000 7ff71dce3700 1 --2- 172.21.15.169:0/211345678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7181035f0 0x7ff718103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:56.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.075+0000 7ff71e4e4700 1 --2- 172.21.15.169:0/211345678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7040035c0 0x7ff7181030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:56.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.076+0000 7ff71ece5700 1 -- 172.21.15.169:0/211345678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7181035f0 msgr2=0x7ff718103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:56.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.076+0000 7ff71ece5700 1 --2- 172.21.15.169:0/211345678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7181035f0 0x7ff718103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:56.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.076+0000 7ff71ece5700 1 -- 172.21.15.169:0/211345678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7040035c0 msgr2=0x7ff7181030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:56.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.076+0000 7ff71ece5700 1 --2- 172.21.15.169:0/211345678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7040035c0 0x7ff7181030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:56.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.076+0000 7ff71ece5700 1 -- 172.21.15.169:0/211345678 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7181a6f20 con 0x7ff7181a4400 2024-04-25T11:58:56.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.076+0000 7ff71ece5700 1 --2- 172.21.15.169:0/211345678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7181a4400 0x7ff7181a67f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff71000ba50 tx=0x7ff71000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:56.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.076+0000 7ff70f7fe700 1 -- 172.21.15.169:0/211345678 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710019070 con 0x7ff7181a4400 2024-04-25T11:58:56.072 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.076+0000 7ff70f7fe700 1 -- 172.21.15.169:0/211345678 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff71000ce00 con 0x7ff7181a4400 2024-04-25T11:58:56.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.076+0000 7ff720748700 1 -- 172.21.15.169:0/211345678 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7181a7210 con 0x7ff7181a4400 2024-04-25T11:58:56.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.076+0000 7ff70f7fe700 1 -- 172.21.15.169:0/211345678 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710007350 con 0x7ff7181a4400 2024-04-25T11:58:56.073 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.076+0000 7ff720748700 1 -- 172.21.15.169:0/211345678 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7181a7700 con 0x7ff7181a4400 2024-04-25T11:58:56.074 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.077+0000 7ff70f7fe700 1 -- 172.21.15.169:0/211345678 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff71001c070 con 0x7ff7181a4400 2024-04-25T11:58:56.074 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.077+0000 7ff720748700 1 -- 172.21.15.169:0/211345678 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7180640a0 con 0x7ff7181a4400 2024-04-25T11:58:56.074 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.077+0000 7ff70f7fe700 1 --2- 172.21.15.169:0/211345678 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff6fc039ae0 0x7ff6fc03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:56.074 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.077+0000 7ff70f7fe700 1 -- 172.21.15.169:0/211345678 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff71004bdb0 con 0x7ff7181a4400 2024-04-25T11:58:56.074 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.078+0000 7ff71e4e4700 1 --2- 172.21.15.169:0/211345678 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff6fc039ae0 0x7ff6fc03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:56.074 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.078+0000 7ff71e4e4700 1 --2- 172.21.15.169:0/211345678 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff6fc039ae0 0x7ff6fc03bfa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff71400f8d0 tx=0x7ff714005e60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:56.076 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.083+0000 7ff70f7fe700 1 -- 172.21.15.169:0/211345678 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff710007df0 con 0x7ff7181a4400 2024-04-25T11:58:56.367 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.373+0000 7ff720748700 1 -- 172.21.15.169:0/211345678 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff7181a7a00 con 0x7ff7181a4400 2024-04-25T11:58:56.368 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:56 smithi169 ceph-mon[32710]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:56.368 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:58:56.368 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:56.368 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.374+0000 7ff70f7fe700 1 -- 172.21.15.169:0/211345678 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7ff710014440 con 0x7ff7181a4400 2024-04-25T11:58:56.369 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-04-25T11:58:56.369 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":3,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","modified":"2024-04-25T11:58:50.062784Z","created":"2024-04-25T11:56:56.493140Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T11:58:56.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.376+0000 7ff720748700 1 -- 172.21.15.169:0/211345678 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff6fc039ae0 msgr2=0x7ff6fc03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:56.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.376+0000 7ff720748700 1 --2- 172.21.15.169:0/211345678 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff6fc039ae0 0x7ff6fc03bfa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff71400f8d0 tx=0x7ff714005e60 comp rx=0 tx=0).stop 2024-04-25T11:58:56.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.376+0000 7ff720748700 1 -- 172.21.15.169:0/211345678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7181a4400 msgr2=0x7ff7181a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:56.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.376+0000 7ff720748700 1 --2- 172.21.15.169:0/211345678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7181a4400 0x7ff7181a67f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff71000ba50 tx=0x7ff71000bd60 comp rx=0 tx=0).stop 2024-04-25T11:58:56.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.376+0000 7ff720748700 1 -- 172.21.15.169:0/211345678 shutdown_connections 2024-04-25T11:58:56.370 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.376+0000 7ff720748700 1 --2- 172.21.15.169:0/211345678 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff6fc039ae0 0x7ff6fc03bfa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:56.371 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.376+0000 7ff720748700 1 --2- 172.21.15.169:0/211345678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7040035c0 0x7ff7181030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:56.371 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.376+0000 7ff720748700 1 --2- 172.21.15.169:0/211345678 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7181a4400 0x7ff7181a67f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:56.371 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.376+0000 7ff720748700 1 --2- 172.21.15.169:0/211345678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7181035f0 0x7ff718103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:56.371 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.376+0000 7ff720748700 1 -- 172.21.15.169:0/211345678 >> 172.21.15.169:0/211345678 conn(0x7ff7180fcf80 msgr2=0x7ff7181068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:56.371 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.376+0000 7ff720748700 1 -- 172.21.15.169:0/211345678 shutdown_connections 2024-04-25T11:58:56.371 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:58:56.376+0000 7ff720748700 1 -- 172.21.15.169:0/211345678 wait complete. 2024-04-25T11:58:56.371 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 3 2024-04-25T11:58:56.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:56 smithi150 ceph-mon[32250]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:56.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:58:56.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:56.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:56 smithi019 ceph-mon[29213]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:56.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:58:56.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:58:57.154 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T11:58:57.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T11:58:57.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: from='client.? 172.21.15.169:0/211345678' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:58:57.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:57.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:58:57.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T11:58:57.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T11:58:57.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T11:58:57.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.401 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: from='client.? 172.21.15.169:0/211345678' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:58:57.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:57.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:58:57.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T11:58:57.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T11:58:57.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T11:58:57.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.405 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:58:57.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: from='client.? 172.21.15.169:0/211345678' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:58:57.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:57.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:58:57.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T11:58:57.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T11:58:57.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T11:58:57.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:57.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:58.481 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:58 smithi019 ceph-mon[29213]: Updating smithi019:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:58.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:58 smithi019 ceph-mon[29213]: Updating smithi150:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:58.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:58 smithi019 ceph-mon[29213]: Updating smithi169:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:58.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:58 smithi019 ceph-mon[29213]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:58.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:58.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:58.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:58.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:58.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:58 smithi019 ceph-mon[29213]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T11:58:58.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:58:58.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:58:58.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:58.482 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:58 smithi019 ceph-mon[29213]: Reconfiguring daemon mon.a on smithi019 2024-04-25T11:58:58.651 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:58 smithi169 ceph-mon[32710]: Updating smithi019:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:58 smithi169 ceph-mon[32710]: Updating smithi150:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:58 smithi169 ceph-mon[32710]: Updating smithi169:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:58 smithi169 ceph-mon[32710]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:58 smithi169 ceph-mon[32710]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T11:58:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:58:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:58:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:58 smithi169 ceph-mon[32710]: Reconfiguring daemon mon.a on smithi019 2024-04-25T11:58:58.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:58 smithi150 ceph-mon[32250]: Updating smithi019:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:58.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:58 smithi150 ceph-mon[32250]: Updating smithi150:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:58.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:58 smithi150 ceph-mon[32250]: Updating smithi169:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:58:58.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:58 smithi150 ceph-mon[32250]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:58:58.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:58.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:58.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:58.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:58:58.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:58 smithi150 ceph-mon[32250]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T11:58:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:58:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:58:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:58 smithi150 ceph-mon[32250]: Reconfiguring daemon mon.a on smithi019 2024-04-25T11:58:58.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.895+0000 7f320b94d700 1 -- 172.21.15.19:0/1283213151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32041084e0 msgr2=0x7f32041088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:58.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.895+0000 7f320b94d700 1 --2- 172.21.15.19:0/1283213151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32041084e0 0x7f32041088c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3200009910 tx=0x7f3200009c20 comp rx=0 tx=0).stop 2024-04-25T11:58:58.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.895+0000 7f320b94d700 1 -- 172.21.15.19:0/1283213151 shutdown_connections 2024-04-25T11:58:58.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.895+0000 7f320b94d700 1 --2- 172.21.15.19:0/1283213151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32041084e0 0x7f32041088c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:58.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.895+0000 7f320b94d700 1 -- 172.21.15.19:0/1283213151 >> 172.21.15.19:0/1283213151 conn(0x7f3204076ea0 msgr2=0x7f32040772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:58.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.895+0000 7f320b94d700 1 -- 172.21.15.19:0/1283213151 shutdown_connections 2024-04-25T11:58:58.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.895+0000 7f320b94d700 1 -- 172.21.15.19:0/1283213151 wait complete. 2024-04-25T11:58:58.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f320b94d700 1 Processor -- start 2024-04-25T11:58:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f320b94d700 1 -- start start 2024-04-25T11:58:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f320b94d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32041084e0 0x7f3204101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f320b94d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32041019a0 0x7f3204101e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f320b94d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32041a43f0 0x7f32041a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f320b94d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f320410ca10 con 0x7f32041a43f0 2024-04-25T11:58:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f320b94d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f320410c890 con 0x7f32041019a0 2024-04-25T11:58:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f320b94d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f320410cb90 con 0x7f32041084e0 2024-04-25T11:58:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f3209eea700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32041a43f0 0x7f32041a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f3209eea700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32041a43f0 0x7f32041a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34896/0 (socket says 172.21.15.19:34896) 2024-04-25T11:58:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f3209eea700 1 -- 172.21.15.19:0/1777653290 learned_addr learned my addr 172.21.15.19:0/1777653290 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:58:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f32096e9700 1 --2- 172.21.15.19:0/1777653290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32041084e0 0x7f3204101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f3208ee8700 1 --2- 172.21.15.19:0/1777653290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32041019a0 0x7f3204101e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f3209eea700 1 -- 172.21.15.19:0/1777653290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32041084e0 msgr2=0x7f3204101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f3209eea700 1 --2- 172.21.15.19:0/1777653290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32041084e0 0x7f3204101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.896+0000 7f3209eea700 1 -- 172.21.15.19:0/1777653290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32041019a0 msgr2=0x7f3204101e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.897+0000 7f3209eea700 1 --2- 172.21.15.19:0/1777653290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32041019a0 0x7f3204101e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.897+0000 7f3209eea700 1 -- 172.21.15.19:0/1777653290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32041a6f10 con 0x7f32041a43f0 2024-04-25T11:58:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.897+0000 7f3209eea700 1 --2- 172.21.15.19:0/1777653290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32041a43f0 0x7f32041a67e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00ba50 tx=0x7f31fc00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.897+0000 7f31f67fc700 1 -- 172.21.15.19:0/1777653290 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc019070 con 0x7f32041a43f0 2024-04-25T11:58:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.897+0000 7f31f67fc700 1 -- 172.21.15.19:0/1777653290 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f31fc014440 con 0x7f32041a43f0 2024-04-25T11:58:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.897+0000 7f31f67fc700 1 -- 172.21.15.19:0/1777653290 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc007570 con 0x7f32041a43f0 2024-04-25T11:58:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.897+0000 7f320b94d700 1 -- 172.21.15.19:0/1777653290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32041a7200 con 0x7f32041a43f0 2024-04-25T11:58:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.897+0000 7f320b94d700 1 -- 172.21.15.19:0/1777653290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32041a76f0 con 0x7f32041a43f0 2024-04-25T11:58:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.898+0000 7f31f67fc700 1 -- 172.21.15.19:0/1777653290 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f31fc01c070 con 0x7f32041a43f0 2024-04-25T11:58:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.898+0000 7f320b94d700 1 -- 172.21.15.19:0/1777653290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32040640a0 con 0x7f32041a43f0 2024-04-25T11:58:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.898+0000 7f31f67fc700 1 --2- 172.21.15.19:0/1777653290 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31f0039ae0 0x7f31f003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:58:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.898+0000 7f31f67fc700 1 -- 172.21.15.19:0/1777653290 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f31fc04bf40 con 0x7f32041a43f0 2024-04-25T11:58:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.898+0000 7f32096e9700 1 --2- 172.21.15.19:0/1777653290 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31f0039ae0 0x7f31f003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:58:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.899+0000 7f32096e9700 1 --2- 172.21.15.19:0/1777653290 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31f0039ae0 0x7f31f003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3200009750 tx=0x7f320000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:58:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:58.903+0000 7f31f67fc700 1 -- 172.21.15.19:0/1777653290 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31fc01b2c0 con 0x7f32041a43f0 2024-04-25T11:58:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:59.116+0000 7f320b94d700 1 -- 172.21.15.19:0/1777653290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f32041a79b0 con 0x7f32041a43f0 2024-04-25T11:58:59.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:59.116+0000 7f31f67fc700 1 -- 172.21.15.19:0/1777653290 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f31fc010ca0 con 0x7f32041a43f0 2024-04-25T11:58:59.109 INFO:teuthology.orchestra.run.smithi019.stdout:# minimal ceph.conf for be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:58:59.109 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-04-25T11:58:59.110 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T11:58:59.110 INFO:teuthology.orchestra.run.smithi019.stdout: mon_host = [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] 2024-04-25T11:58:59.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:59.118+0000 7f320b94d700 1 -- 172.21.15.19:0/1777653290 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31f0039ae0 msgr2=0x7f31f003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:59.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:59.118+0000 7f320b94d700 1 --2- 172.21.15.19:0/1777653290 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31f0039ae0 0x7f31f003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3200009750 tx=0x7f320000b540 comp rx=0 tx=0).stop 2024-04-25T11:58:59.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:59.118+0000 7f320b94d700 1 -- 172.21.15.19:0/1777653290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32041a43f0 msgr2=0x7f32041a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:58:59.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:59.118+0000 7f320b94d700 1 --2- 172.21.15.19:0/1777653290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32041a43f0 0x7f32041a67e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00ba50 tx=0x7f31fc00be10 comp rx=0 tx=0).stop 2024-04-25T11:58:59.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:59.118+0000 7f320b94d700 1 -- 172.21.15.19:0/1777653290 shutdown_connections 2024-04-25T11:58:59.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:59.118+0000 7f320b94d700 1 --2- 172.21.15.19:0/1777653290 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31f0039ae0 0x7f31f003bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:59.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:59.118+0000 7f320b94d700 1 --2- 172.21.15.19:0/1777653290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32041a43f0 0x7f32041a67e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:59.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:59.118+0000 7f320b94d700 1 --2- 172.21.15.19:0/1777653290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32041084e0 0x7f3204101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:59.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:59.118+0000 7f320b94d700 1 --2- 172.21.15.19:0/1777653290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32041019a0 0x7f3204101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:58:59.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:59.118+0000 7f320b94d700 1 -- 172.21.15.19:0/1777653290 >> 172.21.15.19:0/1777653290 conn(0x7f3204076ea0 msgr2=0x7f32040fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:58:59.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:59.118+0000 7f320b94d700 1 -- 172.21.15.19:0/1777653290 shutdown_connections 2024-04-25T11:58:59.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:58:59.118+0000 7f320b94d700 1 -- 172.21.15.19:0/1777653290 wait complete. 2024-04-25T11:58:59.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:58:59 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1777653290' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:59.651 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:58:59 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1777653290' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:59.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:58:59 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1777653290' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:58:59.785 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T11:58:59.786 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:58:59.786 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T11:58:59.813 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:58:59.814 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:58:59.880 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T11:58:59.880 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T11:58:59.909 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T11:58:59.909 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:58:59.974 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:58:59.974 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T11:59:00.006 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:59:00.006 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:59:00.074 INFO:tasks.cephadm:Adding mgr.x on smithi019 2024-04-25T11:59:00.074 INFO:tasks.cephadm:Adding mgr.y on smithi150 2024-04-25T11:59:00.074 INFO:tasks.cephadm:Adding mgr.z on smithi169 2024-04-25T11:59:00.074 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi019=x;smithi150=y;smithi169=z' 2024-04-25T11:59:00.350 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:59:00.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:00 smithi019 ceph-mon[29213]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:00.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:00.401 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:00 smithi169 ceph-mon[32710]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:00.401 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:00.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:00 smithi150 ceph-mon[32250]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:00.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:01.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.793+0000 7f27804c1700 1 -- 172.21.15.169:0/3321347587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27781025d0 msgr2=0x7f27781029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:01.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.793+0000 7f27804c1700 1 --2- 172.21.15.169:0/3321347587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27781025d0 0x7f27781029f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2768009910 tx=0x7f2768009c20 comp rx=0 tx=0).stop 2024-04-25T11:59:01.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.793+0000 7f27804c1700 1 -- 172.21.15.169:0/3321347587 shutdown_connections 2024-04-25T11:59:01.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.793+0000 7f27804c1700 1 --2- 172.21.15.169:0/3321347587 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27781025d0 0x7f27781029f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:01.788 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.793+0000 7f27804c1700 1 -- 172.21.15.169:0/3321347587 >> 172.21.15.169:0/3321347587 conn(0x7f27780fdb70 msgr2=0x7f27780fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:59:01.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.793+0000 7f27804c1700 1 -- 172.21.15.169:0/3321347587 shutdown_connections 2024-04-25T11:59:01.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.794+0000 7f27804c1700 1 -- 172.21.15.169:0/3321347587 wait complete. 2024-04-25T11:59:01.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.794+0000 7f27804c1700 1 Processor -- start 2024-04-25T11:59:01.789 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.794+0000 7f27804c1700 1 -- start start 2024-04-25T11:59:01.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.794+0000 7f27804c1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27781025d0 0x7f27780767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:01.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.794+0000 7f27804c1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2778074e30 0x7f27780752b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:01.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f27804c1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27780757f0 0x7f27781a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:01.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f27804c1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2778108650 con 0x7f27781025d0 2024-04-25T11:59:01.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f27804c1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f27781084d0 con 0x7f27780757f0 2024-04-25T11:59:01.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f27804c1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f27781087d0 con 0x7f2778074e30 2024-04-25T11:59:01.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f277da5c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2778074e30 0x7f27780752b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:01.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f277da5c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2778074e30 0x7f27780752b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:49888/0 (socket says 172.21.15.169:49888) 2024-04-25T11:59:01.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f277da5c700 1 -- 172.21.15.169:0/3049546625 learned_addr learned my addr 172.21.15.169:0/3049546625 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:59:01.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f277e25d700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27781025d0 0x7f27780767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:01.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f277ea5e700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27780757f0 0x7f27781a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:01.791 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f277da5c700 1 -- 172.21.15.169:0/3049546625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27780757f0 msgr2=0x7f27781a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:01.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f277da5c700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27780757f0 0x7f27781a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:01.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f277da5c700 1 -- 172.21.15.169:0/3049546625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27781025d0 msgr2=0x7f27780767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:01.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f277da5c700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27781025d0 0x7f27780767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:01.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f277da5c700 1 -- 172.21.15.169:0/3049546625 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27781a2400 con 0x7f2778074e30 2024-04-25T11:59:01.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f277e25d700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27781025d0 0x7f27780767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:59:01.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f277ea5e700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27780757f0 0x7f27781a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:59:01.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f277da5c700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2778074e30 0x7f27780752b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f277400b780 tx=0x7f277400bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:59:01.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f276f7fe700 1 -- 172.21.15.169:0/3049546625 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774004230 con 0x7f2778074e30 2024-04-25T11:59:01.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f27804c1700 1 -- 172.21.15.169:0/3049546625 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27781a26f0 con 0x7f2778074e30 2024-04-25T11:59:01.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f276f7fe700 1 -- 172.21.15.169:0/3049546625 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f27740043d0 con 0x7f2778074e30 2024-04-25T11:59:01.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f276f7fe700 1 -- 172.21.15.169:0/3049546625 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774011c30 con 0x7f2778074e30 2024-04-25T11:59:01.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.795+0000 7f27804c1700 1 -- 172.21.15.169:0/3049546625 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27781a2cb0 con 0x7f2778074e30 2024-04-25T11:59:01.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.796+0000 7f276f7fe700 1 -- 172.21.15.169:0/3049546625 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f277401a400 con 0x7f2778074e30 2024-04-25T11:59:01.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.796+0000 7f27804c1700 1 -- 172.21.15.169:0/3049546625 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27780640a0 con 0x7f2778074e30 2024-04-25T11:59:01.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.796+0000 7f276f7fe700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2764039ae0 0x7f276403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:01.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.796+0000 7f276f7fe700 1 -- 172.21.15.169:0/3049546625 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f277404c480 con 0x7f2778074e30 2024-04-25T11:59:01.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.797+0000 7f277e25d700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2764039ae0 0x7f276403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:01.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.797+0000 7f277e25d700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2764039ae0 0x7f276403bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2768005360 tx=0x7f276801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:59:01.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:01.802+0000 7f276f7fe700 1 -- 172.21.15.169:0/3049546625 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f277401a840 con 0x7f2778074e30 2024-04-25T11:59:01.901 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:01.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:01.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:59:01.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:59:01.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:01.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:01.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:01.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:59:01.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:59:01.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:02.006 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:02.011+0000 7f27804c1700 1 -- 172.21.15.169:0/3049546625 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi150=y;smithi169=z", "target": ["mon-mgr", ""]}) -- 0x7f2778062f10 con 0x7f2764039ae0 2024-04-25T11:59:02.012 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:02.018+0000 7f276f7fe700 1 -- 172.21.15.169:0/3049546625 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2778062f10 con 0x7f2764039ae0 2024-04-25T11:59:02.012 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mgr update... 2024-04-25T11:59:02.014 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:02.020+0000 7f27804c1700 1 -- 172.21.15.169:0/3049546625 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2764039ae0 msgr2=0x7f276403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:02.014 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:02.020+0000 7f27804c1700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2764039ae0 0x7f276403bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2768005360 tx=0x7f276801a040 comp rx=0 tx=0).stop 2024-04-25T11:59:02.014 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:02.020+0000 7f27804c1700 1 -- 172.21.15.169:0/3049546625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2778074e30 msgr2=0x7f27780752b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:02.015 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:02.020+0000 7f27804c1700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2778074e30 0x7f27780752b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f277400b780 tx=0x7f277400bb40 comp rx=0 tx=0).stop 2024-04-25T11:59:02.015 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:02.020+0000 7f27804c1700 1 -- 172.21.15.169:0/3049546625 shutdown_connections 2024-04-25T11:59:02.015 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:02.020+0000 7f27804c1700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2764039ae0 0x7f276403bfa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:02.015 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:02.020+0000 7f27804c1700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27781025d0 0x7f27780767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:02.015 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:02.020+0000 7f27804c1700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27780757f0 0x7f27781a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:02.015 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:02.020+0000 7f27804c1700 1 --2- 172.21.15.169:0/3049546625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2778074e30 0x7f27780752b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:02.015 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:02.020+0000 7f27804c1700 1 -- 172.21.15.169:0/3049546625 >> 172.21.15.169:0/3049546625 conn(0x7f27780fdb70 msgr2=0x7f27780fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:59:02.015 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:02.020+0000 7f27804c1700 1 -- 172.21.15.169:0/3049546625 shutdown_connections 2024-04-25T11:59:02.015 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:02.020+0000 7f27804c1700 1 -- 172.21.15.169:0/3049546625 wait complete. 2024-04-25T11:59:02.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:02.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:02.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:59:02.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:59:02.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:02.543 DEBUG:teuthology.orchestra.run.smithi150:mgr.y> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T11:59:02.545 DEBUG:teuthology.orchestra.run.smithi169:mgr.z> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T11:59:02.547 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T11:59:02.547 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T11:59:02.547 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:59:02.567 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:59:02.567 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-04-25T11:59:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:59:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:59:02.625 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T11:59:02.625 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:59:02.625 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:59:02.625 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 11:58:07.120639597 +0000 2024-04-25T11:59:02.625 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 11:53:20.595501480 +0000 2024-04-25T11:59:02.626 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 11:53:20.595501480 +0000 2024-04-25T11:59:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 11:53:20.595501480 +0000 2024-04-25T11:59:02.626 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:59:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T11:59:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T11:59:02.694 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000230959 s, 2.2 MB/s 2024-04-25T11:59:02.695 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:59:02.753 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-04-25T11:59:02.810 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:59:02.811 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:59:02.811 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T11:59:02.811 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:59:02.811 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:59:02.811 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 11:58:07.167639118 +0000 2024-04-25T11:59:02.811 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 11:53:20.937498048 +0000 2024-04-25T11:59:02.811 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 11:53:20.937498048 +0000 2024-04-25T11:59:02.811 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 11:53:20.937498048 +0000 2024-04-25T11:59:02.811 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:59:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T11:59:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T11:59:02.876 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000234767 s, 2.2 MB/s 2024-04-25T11:59:02.877 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:59:02.901 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:02 smithi169 ceph-mon[32710]: Reconfiguring mon.b (monmap changed)... 2024-04-25T11:59:02.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:02 smithi169 ceph-mon[32710]: Reconfiguring daemon mon.b on smithi150 2024-04-25T11:59:02.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:02 smithi169 ceph-mon[32710]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:02.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:02 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:02.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:02 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:02.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:02 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:02.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:02 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:59:02.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:02 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:59:02.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:02 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:02.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:02 smithi150 ceph-mon[32250]: Reconfiguring mon.b (monmap changed)... 2024-04-25T11:59:02.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:02 smithi150 ceph-mon[32250]: Reconfiguring daemon mon.b on smithi150 2024-04-25T11:59:02.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:02 smithi150 ceph-mon[32250]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:02.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:02 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:02.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:02 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:02.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:02 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:02.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:02 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:59:02.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:02 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:59:02.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:02 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:02.934 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-04-25T11:59:02.991 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:59:02.991 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:59:02.991 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T11:59:02.991 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:59:02.991 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:59:02.991 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 11:58:07.201638772 +0000 2024-04-25T11:59:02.991 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 11:53:21.272494686 +0000 2024-04-25T11:59:02.992 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 11:53:21.272494686 +0000 2024-04-25T11:59:02.992 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 11:53:21.272494686 +0000 2024-04-25T11:59:02.992 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:59:03.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:02 smithi019 ceph-mon[29213]: Reconfiguring mon.b (monmap changed)... 2024-04-25T11:59:03.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:02 smithi019 ceph-mon[29213]: Reconfiguring daemon mon.b on smithi150 2024-04-25T11:59:03.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:02 smithi019 ceph-mon[29213]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:03.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:02 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:03.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:02 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:03.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:02 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:03.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:02 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:59:03.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:02 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:59:03.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:02 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T11:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T11:59:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000210556 s, 2.4 MB/s 2024-04-25T11:59:03.057 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:59:03.114 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-04-25T11:59:03.170 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:59:03.170 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:59:03.170 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-25T11:59:03.170 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:59:03.170 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:59:03.170 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-04-25 11:58:07.234638435 +0000 2024-04-25T11:59:03.171 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-04-25 11:53:21.621491185 +0000 2024-04-25T11:59:03.171 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-04-25 11:53:21.621491185 +0000 2024-04-25T11:59:03.171 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-04-25 11:53:21.621491185 +0000 2024-04-25T11:59:03.171 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:59:03.233 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-04-25T11:59:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-04-25T11:59:03.234 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000239552 s, 2.1 MB/s 2024-04-25T11:59:03.235 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:59:03.292 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-25T11:59:03.292 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:59:03.310 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:59:03.311 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-04-25T11:59:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:59:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:59:03.369 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T11:59:03.369 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:59:03.369 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:59:03.369 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-25 11:58:49.459942739 +0000 2024-04-25T11:59:03.369 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-25 11:53:34.242287879 +0000 2024-04-25T11:59:03.369 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-25 11:53:34.242287879 +0000 2024-04-25T11:59:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-25 11:53:34.242287879 +0000 2024-04-25T11:59:03.370 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:59:03.434 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-25T11:59:03.434 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-25T11:59:03.435 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000287494 s, 1.8 MB/s 2024-04-25T11:59:03.436 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:59:03.493 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-04-25T11:59:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:59:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:59:03.549 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T11:59:03.549 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:59:03.549 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:59:03.549 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-25 11:58:49.460942720 +0000 2024-04-25T11:59:03.549 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-25 11:53:34.607280692 +0000 2024-04-25T11:59:03.549 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-25 11:53:34.607280692 +0000 2024-04-25T11:59:03.549 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-25 11:53:34.607280692 +0000 2024-04-25T11:59:03.549 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:59:03.613 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-25T11:59:03.613 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-25T11:59:03.613 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000282694 s, 1.8 MB/s 2024-04-25T11:59:03.615 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:59:03.671 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-04-25T11:59:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:59:03.727 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:59:03.728 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T11:59:03.728 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:59:03.728 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:59:03.728 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-25 11:58:49.460942720 +0000 2024-04-25T11:59:03.728 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-25 11:53:34.965273646 +0000 2024-04-25T11:59:03.728 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-25 11:53:34.965273646 +0000 2024-04-25T11:59:03.728 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-25 11:53:34.965273646 +0000 2024-04-25T11:59:03.728 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:59:03.790 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-25T11:59:03.790 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-25T11:59:03.790 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000226009 s, 2.3 MB/s 2024-04-25T11:59:03.791 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:59:03.847 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-04-25T11:59:03.901 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:03 smithi169 ceph-mon[32710]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi150=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:59:03.901 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:03 smithi169 ceph-mon[32710]: Saving service mgr spec with placement smithi019=x;smithi150=y;smithi169=z;count:3 2024-04-25T11:59:03.901 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:03 smithi169 ceph-mon[32710]: Reconfiguring mon.c (monmap changed)... 2024-04-25T11:59:03.901 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:03 smithi169 ceph-mon[32710]: Reconfiguring daemon mon.c on smithi169 2024-04-25T11:59:03.902 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:59:03.902 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:59:03.903 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T11:59:03.903 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:59:03.903 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:59:03.903 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-25 11:58:49.460942720 +0000 2024-04-25T11:59:03.903 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-25 11:53:35.304266974 +0000 2024-04-25T11:59:03.903 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-25 11:53:35.304266974 +0000 2024-04-25T11:59:03.903 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-25 11:53:35.304266974 +0000 2024-04-25T11:59:03.903 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:59:03.962 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:03 smithi150 ceph-mon[32250]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi150=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:59:03.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:03 smithi150 ceph-mon[32250]: Saving service mgr spec with placement smithi019=x;smithi150=y;smithi169=z;count:3 2024-04-25T11:59:03.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:03 smithi150 ceph-mon[32250]: Reconfiguring mon.c (monmap changed)... 2024-04-25T11:59:03.963 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:03 smithi150 ceph-mon[32250]: Reconfiguring daemon mon.c on smithi169 2024-04-25T11:59:03.965 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-25T11:59:03.965 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-25T11:59:03.965 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000231529 s, 2.2 MB/s 2024-04-25T11:59:03.966 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:59:04.022 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-04-25T11:59:04.022 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:59:04.038 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:59:04.038 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-04-25T11:59:04.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:03 smithi019 ceph-mon[29213]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi150=y;smithi169=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:59:04.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:03 smithi019 ceph-mon[29213]: Saving service mgr spec with placement smithi019=x;smithi150=y;smithi169=z;count:3 2024-04-25T11:59:04.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:03 smithi019 ceph-mon[29213]: Reconfiguring mon.c (monmap changed)... 2024-04-25T11:59:04.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:03 smithi019 ceph-mon[29213]: Reconfiguring daemon mon.c on smithi169 2024-04-25T11:59:04.093 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:59:04.093 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:59:04.093 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T11:59:04.093 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:59:04.093 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:59:04.094 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 11:58:40.164625141 +0000 2024-04-25T11:59:04.094 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 11:53:34.105674486 +0000 2024-04-25T11:59:04.094 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 11:53:34.105674486 +0000 2024-04-25T11:59:04.094 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 11:53:34.105674486 +0000 2024-04-25T11:59:04.094 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:59:04.157 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T11:59:04.157 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T11:59:04.158 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000346274 s, 1.5 MB/s 2024-04-25T11:59:04.159 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:59:04.214 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-04-25T11:59:04.269 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:59:04.269 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:59:04.269 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T11:59:04.270 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:59:04.270 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:59:04.270 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 11:58:40.164625141 +0000 2024-04-25T11:59:04.270 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 11:53:34.450666921 +0000 2024-04-25T11:59:04.270 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 11:53:34.450666921 +0000 2024-04-25T11:59:04.270 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 11:53:34.450666921 +0000 2024-04-25T11:59:04.270 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:59:04.332 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T11:59:04.332 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T11:59:04.332 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.00027276 s, 1.9 MB/s 2024-04-25T11:59:04.333 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:59:04.389 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-04-25T11:59:04.444 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:59:04.445 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:59:04.445 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T11:59:04.445 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:59:04.445 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:59:04.445 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 11:58:40.165625120 +0000 2024-04-25T11:59:04.445 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 11:53:34.803659180 +0000 2024-04-25T11:59:04.445 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 11:53:34.803659180 +0000 2024-04-25T11:59:04.445 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 11:53:34.803659180 +0000 2024-04-25T11:59:04.445 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:59:04.507 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T11:59:04.507 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T11:59:04.507 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000229836 s, 2.2 MB/s 2024-04-25T11:59:04.509 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:59:04.563 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-04-25T11:59:04.619 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:59:04.619 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:59:04.619 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T11:59:04.619 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:59:04.619 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:59:04.619 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-04-25 11:58:40.165625120 +0000 2024-04-25T11:59:04.619 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-04-25 11:53:35.148651615 +0000 2024-04-25T11:59:04.619 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-04-25 11:53:35.148651615 +0000 2024-04-25T11:59:04.619 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-04-25 11:53:35.148651615 +0000 2024-04-25T11:59:04.619 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:59:04.681 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-04-25T11:59:04.681 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-04-25T11:59:04.681 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000202913 s, 2.5 MB/s 2024-04-25T11:59:04.682 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:59:04.738 INFO:tasks.cephadm:Deploying osd.0 on smithi019 with /dev/vg_nvme/lv_4... 2024-04-25T11:59:04.739 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T11:59:04.901 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:04 smithi169 ceph-mon[32710]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:04.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:04 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:04.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:04 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:04.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:04 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:04.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:04 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:04.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:04 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:04.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:04 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:04.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:04 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:59:04.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:04 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:59:04.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:04 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:59:04.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:04 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:04.982 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:59:05.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:04 smithi019 ceph-mon[29213]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:05.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:04 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:05.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:04 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:05.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:04 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:05.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:04 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:05.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:04 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:05.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:04 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:05.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:04 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:59:05.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:04 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:59:05.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:04 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:59:05.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:04 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:05.159 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:04 smithi150 ceph-mon[32250]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:05.159 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:04 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:05.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:04 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:05.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:04 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:05.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:04 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:05.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:04 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:05.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:04 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:05.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:04 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:59:05.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:04 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:59:05.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:04 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:59:05.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:04 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:05.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:05 smithi019 ceph-mon[29213]: Deploying daemon mgr.z on smithi169 2024-04-25T11:59:06.035 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:05 smithi169 ceph-mon[32710]: Deploying daemon mgr.z on smithi169 2024-04-25T11:59:06.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:05 smithi150 ceph-mon[32250]: Deploying daemon mgr.z on smithi169 2024-04-25T11:59:06.652 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:06 smithi169 systemd[1]: Starting Ceph mgr.z for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T11:59:06.866 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T11:59:06.867 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T11:59:06.867 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-25T11:59:06.867 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-25T11:59:06.867 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113445 s, 924 MB/s 2024-04-25T11:59:06.867 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-25T11:59:06.949 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:06 smithi169 ceph-mon[32710]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:06.950 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:06 smithi169 podman[34725]: 2024-04-25 11:59:06.59867099 +0000 UTC m=+0.018142486 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:59:06.950 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:06 smithi169 podman[34725]: 2024-04-25 11:59:06.70236122 +0000 UTC m=+0.121832702 container create 631bf7fcd2a3d1c24ea2a6042b2a5f3a009681b198f6f0965af1227aef2bb073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T11:59:06.950 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:06 smithi169 podman[34725]: 2024-04-25 11:59:06.929602726 +0000 UTC m=+0.349074208 container init 631bf7fcd2a3d1c24ea2a6042b2a5f3a009681b198f6f0965af1227aef2bb073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T11:59:06.950 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:06 smithi169 podman[34725]: 2024-04-25 11:59:06.932141633 +0000 UTC m=+0.351613125 container start 631bf7fcd2a3d1c24ea2a6042b2a5f3a009681b198f6f0965af1227aef2bb073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T11:59:06.950 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:06 smithi169 ceph-mgr[34744]: -- 172.21.15.169:0/792340747 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df627d45a0 con 0x55df627a8800 2024-04-25T11:59:07.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:06 smithi019 ceph-mon[29213]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:07.159 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:06 smithi150 ceph-mon[32250]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:07.261 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:07 smithi169 bash[34725]: 631bf7fcd2a3d1c24ea2a6042b2a5f3a009681b198f6f0965af1227aef2bb073 2024-04-25T11:59:07.261 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:07 smithi169 systemd[1]: Started Ceph mgr.z for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T11:59:07.261 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:07.083+0000 7fbbd9577200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:59:07.261 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:07.267+0000 7fbbd9577200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:59:07.444 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_4 2024-04-25T11:59:07.651 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:07.367+0000 7fbbd9577200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:59:07.686 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:59:08.077 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:07.674+0000 7fbbd9577200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:59:08.077 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:07.760+0000 7fbbd9577200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:59:08.401 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:08 smithi169 ceph-mon[32710]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:08.401 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:08.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:08.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:08.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:59:08.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:59:08.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:59:08.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:08.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:08 smithi169 ceph-mon[32710]: Deploying daemon mgr.y on smithi150 2024-04-25T11:59:08.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:08 smithi150 ceph-mon[32250]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:08.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:08.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:08.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:08.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:59:08.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:59:08.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:59:08.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:08.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:08 smithi150 ceph-mon[32250]: Deploying daemon mgr.y on smithi150 2024-04-25T11:59:08.473 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:08 smithi019 ceph-mon[29213]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:08.473 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:08.473 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:08.473 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:08.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:59:08.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:59:08.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:59:08.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:08.474 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:08 smithi019 ceph-mon[29213]: Deploying daemon mgr.y on smithi150 2024-04-25T11:59:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.242+0000 7f46d3859700 1 -- 172.21.15.19:0/1065815345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc102aa0 msgr2=0x7f46cc102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.242+0000 7f46d3859700 1 --2- 172.21.15.19:0/1065815345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc102aa0 0x7f46cc102ec0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f46c80099c0 tx=0x7f46c8009cd0 comp rx=0 tx=0).stop 2024-04-25T11:59:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 -- 172.21.15.19:0/1065815345 shutdown_connections 2024-04-25T11:59:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 --2- 172.21.15.19:0/1065815345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc102aa0 0x7f46cc102ec0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 --2- 172.21.15.19:0/1065815345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46cc104640 0x7f46cc10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 --2- 172.21.15.19:0/1065815345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46cc103ca0 0x7f46cc104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 -- 172.21.15.19:0/1065815345 >> 172.21.15.19:0/1065815345 conn(0x7f46cc0fe040 msgr2=0x7f46cc100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:59:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 -- 172.21.15.19:0/1065815345 shutdown_connections 2024-04-25T11:59:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 -- 172.21.15.19:0/1065815345 wait complete. 2024-04-25T11:59:09.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 Processor -- start 2024-04-25T11:59:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 -- start start 2024-04-25T11:59:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46cc102aa0 0x7f46cc0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc103ca0 0x7f46cc074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46cc104640 0x7f46cc075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc10e6c0 con 0x7f46cc103ca0 2024-04-25T11:59:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc10e540 con 0x7f46cc104640 2024-04-25T11:59:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.243+0000 7f46d3859700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc10e840 con 0x7f46cc102aa0 2024-04-25T11:59:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46d0df4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc103ca0 0x7f46cc074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46d0df4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc103ca0 0x7f46cc074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58650/0 (socket says 172.21.15.19:58650) 2024-04-25T11:59:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46d0df4700 1 -- 172.21.15.19:0/1343258687 learned_addr learned my addr 172.21.15.19:0/1343258687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T11:59:09.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46d15f5700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46cc102aa0 0x7f46cc0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46d1df6700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46cc104640 0x7f46cc075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46d0df4700 1 -- 172.21.15.19:0/1343258687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46cc102aa0 msgr2=0x7f46cc0767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46d0df4700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46cc102aa0 0x7f46cc0767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46d0df4700 1 -- 172.21.15.19:0/1343258687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46cc104640 msgr2=0x7f46cc075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46d0df4700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46cc104640 0x7f46cc075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46d0df4700 1 -- 172.21.15.19:0/1343258687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc075c20 con 0x7f46cc103ca0 2024-04-25T11:59:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46d1df6700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46cc104640 0x7f46cc075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:59:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46d0df4700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc103ca0 0x7f46cc074e30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00d980 tx=0x7f46bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:59:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46c27fc700 1 -- 172.21.15.19:0/1343258687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc016070 con 0x7f46cc103ca0 2024-04-25T11:59:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46d3859700 1 -- 172.21.15.19:0/1343258687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc1a2eb0 con 0x7f46cc103ca0 2024-04-25T11:59:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46c27fc700 1 -- 172.21.15.19:0/1343258687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f46bc004d00 con 0x7f46cc103ca0 2024-04-25T11:59:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46c27fc700 1 -- 172.21.15.19:0/1343258687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc005610 con 0x7f46cc103ca0 2024-04-25T11:59:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.244+0000 7f46d3859700 1 -- 172.21.15.19:0/1343258687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc1a3310 con 0x7f46cc103ca0 2024-04-25T11:59:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.245+0000 7f46c27fc700 1 -- 172.21.15.19:0/1343258687 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f46bc004830 con 0x7f46cc103ca0 2024-04-25T11:59:09.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.245+0000 7f46d3859700 1 -- 172.21.15.19:0/1343258687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46cc069730 con 0x7f46cc103ca0 2024-04-25T11:59:09.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.245+0000 7f46c27fc700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f46b8039890 0x7f46b803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:09.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.245+0000 7f46c27fc700 1 -- 172.21.15.19:0/1343258687 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f46bc04c870 con 0x7f46cc103ca0 2024-04-25T11:59:09.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.245+0000 7f46d15f5700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f46b8039890 0x7f46b803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:09.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.246+0000 7f46d15f5700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f46b8039890 0x7f46b803bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f46c80038a0 tx=0x7f46c8005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:59:09.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.250+0000 7f46c27fc700 1 -- 172.21.15.19:0/1343258687 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46bc01f020 con 0x7f46cc103ca0 2024-04-25T11:59:09.288 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:08 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:08.953+0000 7fbbd9577200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:59:09.289 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:09.122+0000 7fbbd9577200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:59:09.289 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:09.294+0000 7fbbd9577200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:59:09.370 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:09 smithi150 systemd[1]: Starting Ceph mgr.y for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T11:59:09.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:09.470+0000 7f46d3859700 1 -- 172.21.15.19:0/1343258687 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f46cc062e80 con 0x7f46b8039890 2024-04-25T11:59:09.659 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:09 smithi150 podman[33631]: 2024-04-25 11:59:09.279289798 +0000 UTC m=+0.017920299 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:59:09.659 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:09 smithi150 podman[33631]: 2024-04-25 11:59:09.387277714 +0000 UTC m=+0.125908204 container create 77cdcfc1595191b304d04177090607de609101c6de48440ca8ded64e4b179a27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T11:59:09.659 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:09 smithi150 podman[33631]: 2024-04-25 11:59:09.597766545 +0000 UTC m=+0.336397036 container init 77cdcfc1595191b304d04177090607de609101c6de48440ca8ded64e4b179a27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T11:59:09.660 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:09 smithi150 podman[33631]: 2024-04-25 11:59:09.600287399 +0000 UTC m=+0.338917893 container start 77cdcfc1595191b304d04177090607de609101c6de48440ca8ded64e4b179a27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T11:59:09.660 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:09 smithi150 ceph-mgr[33650]: -- 172.21.15.150:0/1934718093 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55929b47e5a0 con 0x55929b453000 2024-04-25T11:59:09.869 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:09.578+0000 7fbbd9577200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:59:09.869 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:09.670+0000 7fbbd9577200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:59:09.869 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:09.875+0000 7fbbd9577200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:59:09.938 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:09 smithi150 bash[33631]: 77cdcfc1595191b304d04177090607de609101c6de48440ca8ded64e4b179a27 2024-04-25T11:59:09.938 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:09 smithi150 systemd[1]: Started Ceph mgr.y for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T11:59:09.938 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:09 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:09.753+0000 7f5636625200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:59:09.938 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:09 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:09.943+0000 7f5636625200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:59:10.127 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:10 smithi019 ceph-mon[29213]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:10.127 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:10 smithi019 ceph-mon[29213]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:59:10.127 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:10 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:59:10.127 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:10 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:59:10.127 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:10 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:10.127 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:10 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.127 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:10 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.127 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:10 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.127 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:10 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.127 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:10 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.127 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:10 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:10.151 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:10 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:10.074+0000 7fbbd9577200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:59:10.151 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:10 smithi169 ceph-mon[32710]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:10.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:10 smithi169 ceph-mon[32710]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:59:10.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:10 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:59:10.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:10 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:59:10.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:10 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:10.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:10 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:10 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:10 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:10 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:10 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:10 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:10.370 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:10 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:10.049+0000 7f5636625200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:59:10.370 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:10 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:10.376+0000 7f5636625200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:59:10.370 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:10 smithi150 ceph-mon[32250]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:10.371 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:10 smithi150 ceph-mon[32250]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:59:10.371 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:10 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:59:10.371 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:10 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:59:10.371 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:10 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:10.371 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:10 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.371 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:10 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.371 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:10 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.371 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:10 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.371 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:10 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:10.371 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:10 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:10.659 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:10 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:10.474+0000 7f5636625200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:59:11.148 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:10 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:10.712+0000 7fbbd9577200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:59:11.401 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:11 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:11.154+0000 7fbbd9577200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:59:11.402 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:11 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:11.252+0000 7fbbd9577200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:59:11.901 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:11 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:11.452+0000 7fbbd9577200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:59:12.058 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:11 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:11.725+0000 7f5636625200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:59:12.058 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:11 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:11.892+0000 7f5636625200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:59:12.058 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:12.064+0000 7f5636625200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:59:12.301 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:11 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:11.973+0000 7fbbd9577200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:59:12.301 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:12.065+0000 7fbbd9577200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:59:12.302 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-mon[32710]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:12.302 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.302 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.302 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.302 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.302 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:12.302 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:12.302 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.302 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-mon[32710]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T11:59:12.302 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:59:12.302 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:59:12.302 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:12.303 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-mon[32710]: Reconfiguring daemon mgr.x on smithi019 2024-04-25T11:59:12.345 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-mon[32250]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:12.345 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.345 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.345 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.345 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.345 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:12.345 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:12.345 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.345 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-mon[32250]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T11:59:12.345 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:59:12.346 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:59:12.346 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:12.346 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-mon[32250]: Reconfiguring daemon mgr.x on smithi019 2024-04-25T11:59:12.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:12 smithi019 ceph-mon[29213]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:12.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:12 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:12 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:12 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:12 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:12 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:12.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:12 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:12.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:12 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:12.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:12 smithi019 ceph-mon[29213]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T11:59:12.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:12 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:59:12.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:12 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:59:12.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:12 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:12.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:12 smithi019 ceph-mon[29213]: Reconfiguring daemon mgr.x on smithi019 2024-04-25T11:59:12.640 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:12.350+0000 7f5636625200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:59:12.641 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:12.441+0000 7f5636625200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:59:12.641 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:12.646+0000 7f5636625200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:59:12.643 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:12.307+0000 7fbbd9577200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:59:12.643 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:12.440+0000 7fbbd9577200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:59:12.643 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:12.534+0000 7fbbd9577200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:59:12.643 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:12.648+0000 7fbbd9577200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:59:12.909 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:12 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:12.837+0000 7f5636625200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:59:13.402 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:12.956+0000 7fbbd9577200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:59:13.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:13 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3136560023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73d2b9f6-57f7-4008-9269-75c46243c8e0"}]: dispatch 2024-04-25T11:59:13.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:13 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3136560023' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73d2b9f6-57f7-4008-9269-75c46243c8e0"}]': finished 2024-04-25T11:59:13.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:13 smithi169 ceph-mon[32710]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T11:59:13.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:13 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:13.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:13 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1009704111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:59:13.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:13 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3136560023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73d2b9f6-57f7-4008-9269-75c46243c8e0"}]: dispatch 2024-04-25T11:59:13.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:13 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3136560023' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73d2b9f6-57f7-4008-9269-75c46243c8e0"}]': finished 2024-04-25T11:59:13.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:13 smithi019 ceph-mon[29213]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T11:59:13.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:13 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:13.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:13 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1009704111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:59:13.659 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:13 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:13.437+0000 7f5636625200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:59:13.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:13 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3136560023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73d2b9f6-57f7-4008-9269-75c46243c8e0"}]: dispatch 2024-04-25T11:59:13.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:13 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3136560023' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73d2b9f6-57f7-4008-9269-75c46243c8e0"}]': finished 2024-04-25T11:59:13.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:13 smithi150 ceph-mon[32250]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T11:59:13.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:13 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:13.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:13 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1009704111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:59:13.901 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:13 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:13.552+0000 7fbbd9577200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:59:13.902 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 11:59:13 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T11:59:13.764+0000 7fbbd9577200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:59:14.140 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:13 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:13.851+0000 7f5636625200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:59:14.141 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:13 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:13.944+0000 7f5636625200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:59:14.141 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:14 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:14.145+0000 7f5636625200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:59:14.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:14.249+0000 7f46c27fc700 1 -- 172.21.15.19:0/1343258687 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f46bc003680 con 0x7f46cc103ca0 2024-04-25T11:59:14.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:14 smithi150 ceph-mon[32250]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:14.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:14 smithi150 ceph-mon[32250]: Standby manager daemon z started 2024-04-25T11:59:14.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:14 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.169:0/3589907794' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T11:59:14.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:14 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.169:0/3589907794' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:59:14.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:14 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.169:0/3589907794' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T11:59:14.410 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:14 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.169:0/3589907794' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:59:14.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:14 smithi019 ceph-mon[29213]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:14.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:14 smithi019 ceph-mon[29213]: Standby manager daemon z started 2024-04-25T11:59:14.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:14 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.169:0/3589907794' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T11:59:14.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:14 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.169:0/3589907794' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:59:14.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:14 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.169:0/3589907794' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T11:59:14.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:14 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.169:0/3589907794' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:59:14.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:14 smithi169 ceph-mon[32710]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:14.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:14 smithi169 ceph-mon[32710]: Standby manager daemon z started 2024-04-25T11:59:14.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:14 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.169:0/3589907794' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T11:59:14.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:14 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.169:0/3589907794' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:59:14.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:14 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.169:0/3589907794' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T11:59:14.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:14 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.169:0/3589907794' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:59:14.991 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:14 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:14.666+0000 7f5636625200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:59:14.991 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:14 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:14.757+0000 7f5636625200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:59:15.252 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:14 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:14.997+0000 7f5636625200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:59:15.252 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:15 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:15.128+0000 7f5636625200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:59:15.252 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:15 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:15.222+0000 7f5636625200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:59:15.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:15 smithi019 ceph-mon[29213]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-25T11:59:15.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T11:59:15.638 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:15 smithi150 ceph-mon[32250]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-25T11:59:15.638 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T11:59:15.638 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:15 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:15.337+0000 7f5636625200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:59:15.638 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:15 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:15.644+0000 7f5636625200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:59:15.651 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:15 smithi169 ceph-mon[32710]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-25T11:59:15.651 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T11:59:16.510 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:16 smithi169 ceph-mon[32710]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:16.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:16 smithi019 ceph-mon[29213]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:16.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:16 smithi150 ceph-mon[32250]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:16.659 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:16 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:16.242+0000 7f5636625200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:59:16.659 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 11:59:16 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T11:59:16.463+0000 7f5636625200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:59:17.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:17.271+0000 7f46c27fc700 1 -- 172.21.15.19:0/1343258687 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f46bc022350 con 0x7f46cc103ca0 2024-04-25T11:59:17.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:17 smithi019 ceph-mon[29213]: Standby manager daemon y started 2024-04-25T11:59:17.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:17 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.150:0/3139131554' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T11:59:17.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:17 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.150:0/3139131554' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:59:17.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:17 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.150:0/3139131554' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T11:59:17.489 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:17 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.150:0/3139131554' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:59:17.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:17 smithi169 ceph-mon[32710]: Standby manager daemon y started 2024-04-25T11:59:17.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:17 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.150:0/3139131554' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T11:59:17.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:17 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.150:0/3139131554' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:59:17.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:17 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.150:0/3139131554' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T11:59:17.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:17 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.150:0/3139131554' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:59:17.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:17 smithi150 ceph-mon[32250]: Standby manager daemon y started 2024-04-25T11:59:17.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:17 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.150:0/3139131554' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T11:59:17.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:17 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.150:0/3139131554' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:59:17.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:17 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.150:0/3139131554' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T11:59:17.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:17 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.150:0/3139131554' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:59:18.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:18 smithi019 ceph-mon[29213]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:18.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:18 smithi019 ceph-mon[29213]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-04-25T11:59:18.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T11:59:18.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:18.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:18.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:18.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:18.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:18.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:18.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:18 smithi169 ceph-mon[32710]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:18.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:18 smithi169 ceph-mon[32710]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-04-25T11:59:18.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:18 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T11:59:18.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:18 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:18.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:18 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:18.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:18 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:18.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:18 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:18.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:18 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:18.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:18 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:18.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:18 smithi150 ceph-mon[32250]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:18.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:18 smithi150 ceph-mon[32250]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-04-25T11:59:18.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:18 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T11:59:18.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:18 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:18.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:18 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:18.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:18 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:18.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:18 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:18.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:18 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:18.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:18 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:20.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:20 smithi019 ceph-mon[29213]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:20.651 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:20 smithi169 ceph-mon[32710]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:20.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:20 smithi150 ceph-mon[32250]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:21.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T11:59:21.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:21.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:21 smithi019 ceph-mon[29213]: Deploying daemon osd.0 on smithi019 2024-04-25T11:59:21.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T11:59:21.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:21.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:21 smithi169 ceph-mon[32710]: Deploying daemon osd.0 on smithi019 2024-04-25T11:59:21.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T11:59:21.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:21.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:21 smithi150 ceph-mon[32250]: Deploying daemon osd.0 on smithi019 2024-04-25T11:59:22.478 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:22 smithi019 ceph-mon[29213]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:22.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:22 smithi169 ceph-mon[32710]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:22.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:22 smithi150 ceph-mon[32250]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:24.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:24 smithi019 ceph-mon[29213]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:24.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:24 smithi169 ceph-mon[32710]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:24.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:24 smithi150 ceph-mon[32250]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:26.460 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:26 smithi019 ceph-mon[29213]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:26.460 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:26.460 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:26.460 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:26.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:26 smithi169 ceph-mon[32710]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:26.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:26.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:26.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:26.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:26 smithi150 ceph-mon[32250]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:26.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:26.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:26.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:28.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:28 smithi019 ceph-mon[29213]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:28.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:28 smithi169 ceph-mon[32710]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:28.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:28 smithi150 ceph-mon[32250]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:29.521 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 0 on host 'smithi019' 2024-04-25T11:59:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:29.525+0000 7f46c27fc700 1 -- 172.21.15.19:0/1343258687 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f46cc062e80 con 0x7f46b8039890 2024-04-25T11:59:29.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:29.528+0000 7f46d3859700 1 -- 172.21.15.19:0/1343258687 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f46b8039890 msgr2=0x7f46b803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:29.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:29.528+0000 7f46d3859700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f46b8039890 0x7f46b803bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f46c80038a0 tx=0x7f46c8005ad0 comp rx=0 tx=0).stop 2024-04-25T11:59:29.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:29.528+0000 7f46d3859700 1 -- 172.21.15.19:0/1343258687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc103ca0 msgr2=0x7f46cc074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:29.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:29.528+0000 7f46d3859700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc103ca0 0x7f46cc074e30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00d980 tx=0x7f46bc00dc90 comp rx=0 tx=0).stop 2024-04-25T11:59:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:29.528+0000 7f46d3859700 1 -- 172.21.15.19:0/1343258687 shutdown_connections 2024-04-25T11:59:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:29.528+0000 7f46d3859700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f46b8039890 0x7f46b803bd50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:29.528+0000 7f46d3859700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46cc104640 0x7f46cc075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:29.528+0000 7f46d3859700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f46cc102aa0 0x7f46cc0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:29.528+0000 7f46d3859700 1 --2- 172.21.15.19:0/1343258687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46cc103ca0 0x7f46cc074e30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:29.528+0000 7f46d3859700 1 -- 172.21.15.19:0/1343258687 >> 172.21.15.19:0/1343258687 conn(0x7f46cc0fe040 msgr2=0x7f46cc107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:59:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:29.528+0000 7f46d3859700 1 -- 172.21.15.19:0/1343258687 shutdown_connections 2024-04-25T11:59:29.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T11:59:29.528+0000 7f46d3859700 1 -- 172.21.15.19:0/1343258687 wait complete. 2024-04-25T11:59:29.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:29.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:29.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:29.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:29.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:29.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:29.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:29.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:29.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:29.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:29.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:29.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:29.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:29.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:29.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:29.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:29.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:29.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:29.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:29.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:29.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:29.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:29.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:29.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:30.226 DEBUG:teuthology.orchestra.run.smithi019:osd.0> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T11:59:30.228 INFO:tasks.cephadm:Deploying osd.1 on smithi150 with /dev/vg_nvme/lv_4... 2024-04-25T11:59:30.229 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T11:59:30.471 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:59:30.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:30 smithi019 ceph-mon[29213]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:30.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:30 smithi019 ceph-mon[29213]: from='osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T11:59:30.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:30 smithi169 ceph-mon[32710]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:30.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:30 smithi169 ceph-mon[32710]: from='osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T11:59:30.907 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:30 smithi150 ceph-mon[32250]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:30.907 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:30 smithi150 ceph-mon[32250]: from='osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T11:59:31.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:31 smithi019 ceph-mon[29213]: from='osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T11:59:31.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:31 smithi019 ceph-mon[29213]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T11:59:31.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:31 smithi019 ceph-mon[29213]: from='osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T11:59:31.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:31 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:31.696 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:59:31 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-0[44023]: 2024-04-25T11:59:31.642+0000 7fa078d63700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T11:59:31.696 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 11:59:31 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-0[44023]: 2024-04-25T11:59:31.649+0000 7fa074b56700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:59:31.849 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:31 smithi150 ceph-mon[32250]: from='osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T11:59:31.849 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:31 smithi150 ceph-mon[32250]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T11:59:31.849 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:31 smithi150 ceph-mon[32250]: from='osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T11:59:31.849 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:31 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:31.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:31 smithi169 ceph-mon[32710]: from='osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T11:59:31.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:31 smithi169 ceph-mon[32710]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T11:59:31.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:31 smithi169 ceph-mon[32710]: from='osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T11:59:31.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:31 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:32.468 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T11:59:32.468 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T11:59:32.468 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-25T11:59:32.468 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-25T11:59:32.468 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0130233 s, 805 MB/s 2024-04-25T11:59:32.468 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-25T11:59:32.842 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:32 smithi150 ceph-mon[32250]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:32.842 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:32 smithi150 ceph-mon[32250]: from='osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T11:59:32.842 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:32 smithi150 ceph-mon[32250]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T11:59:32.842 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:32 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:32.842 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:32 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:32.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:32 smithi169 ceph-mon[32710]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:32.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:32 smithi169 ceph-mon[32710]: from='osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T11:59:32.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:32 smithi169 ceph-mon[32710]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T11:59:32.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:32 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:32.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:32 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:32.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:32 smithi019 ceph-mon[29213]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:32.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:32 smithi019 ceph-mon[29213]: from='osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T11:59:32.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:32 smithi019 ceph-mon[29213]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T11:59:32.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:32 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:32.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:32 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:33.212 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2024-04-25T11:59:33.465 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:59:33.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:33 smithi169 ceph-mon[32710]: purged_snaps scrub starts 2024-04-25T11:59:33.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:33 smithi169 ceph-mon[32710]: purged_snaps scrub ok 2024-04-25T11:59:33.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:33 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:33.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:33 smithi169 ceph-mon[32710]: osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755] boot 2024-04-25T11:59:33.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:33 smithi169 ceph-mon[32710]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T11:59:33.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:33 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:33.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:33 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:33.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:33 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:33.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:33 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:59:33.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:33 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:33.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:33 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:33.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:33 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:33.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:33 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:33.906 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:33 smithi150 ceph-mon[32250]: purged_snaps scrub starts 2024-04-25T11:59:33.906 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:33 smithi150 ceph-mon[32250]: purged_snaps scrub ok 2024-04-25T11:59:33.906 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:33 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:33.907 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:33 smithi150 ceph-mon[32250]: osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755] boot 2024-04-25T11:59:33.907 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:33 smithi150 ceph-mon[32250]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T11:59:33.907 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:33 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:33.907 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:33 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:33.907 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:33 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:33.907 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:33 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:59:33.907 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:33 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:33.907 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:33 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:33.907 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:33 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:33.907 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:33 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:34.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:33 smithi019 ceph-mon[29213]: purged_snaps scrub starts 2024-04-25T11:59:34.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:33 smithi019 ceph-mon[29213]: purged_snaps scrub ok 2024-04-25T11:59:34.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:34.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:33 smithi019 ceph-mon[29213]: osd.0 [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755] boot 2024-04-25T11:59:34.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:33 smithi019 ceph-mon[29213]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T11:59:34.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:59:34.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:34.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:34.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:59:34.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:34.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:34.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:34.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:33 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:34.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.912+0000 7fd223a2b700 1 -- 172.21.15.150:0/1406579437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd21c108850 msgr2=0x7fd21c10ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:34.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.912+0000 7fd223a2b700 1 --2- 172.21.15.150:0/1406579437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd21c108850 0x7fd21c10ac40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd214009990 tx=0x7fd214009ca0 comp rx=0 tx=0).stop 2024-04-25T11:59:34.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.912+0000 7fd223a2b700 1 -- 172.21.15.150:0/1406579437 shutdown_connections 2024-04-25T11:59:34.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.912+0000 7fd223a2b700 1 --2- 172.21.15.150:0/1406579437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd21c108850 0x7fd21c10ac40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:34.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.912+0000 7fd223a2b700 1 --2- 172.21.15.150:0/1406579437 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd21c1035f0 0x7fd21c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:34.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.912+0000 7fd223a2b700 1 --2- 172.21.15.150:0/1406579437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd21c105f20 0x7fd21c108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:34.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.912+0000 7fd223a2b700 1 -- 172.21.15.150:0/1406579437 >> 172.21.15.150:0/1406579437 conn(0x7fd21c0fcf40 msgr2=0x7fd21c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:59:34.907 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.912+0000 7fd223a2b700 1 -- 172.21.15.150:0/1406579437 shutdown_connections 2024-04-25T11:59:34.908 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.912+0000 7fd223a2b700 1 -- 172.21.15.150:0/1406579437 wait complete. 2024-04-25T11:59:34.908 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:34 smithi150 ceph-mon[32250]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:34.908 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:34 smithi150 ceph-mon[32250]: Detected new or changed devices on smithi019 2024-04-25T11:59:34.908 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:34 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi019 to 17093M 2024-04-25T11:59:34.908 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:34 smithi150 ceph-mon[32250]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T11:59:34.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd223a2b700 1 Processor -- start 2024-04-25T11:59:34.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd223a2b700 1 -- start start 2024-04-25T11:59:34.910 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd223a2b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd21c1035f0 0x7fd21c102f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:34.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd223a2b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd21c105f20 0x7fd21c1015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:34.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd223a2b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd21c108850 0x7fd21c101ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:34.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd223a2b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c10e670 con 0x7fd21c108850 2024-04-25T11:59:34.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd223a2b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c10e4f0 con 0x7fd21c105f20 2024-04-25T11:59:34.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd223a2b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c10e7f0 con 0x7fd21c1035f0 2024-04-25T11:59:34.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd220fc6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd21c105f20 0x7fd21c1015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:34.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd220fc6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd21c105f20 0x7fd21c1015a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55474/0 (socket says 172.21.15.150:55474) 2024-04-25T11:59:34.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd220fc6700 1 -- 172.21.15.150:0/2831429459 learned_addr learned my addr 172.21.15.150:0/2831429459 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T11:59:34.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd2217c7700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd21c1035f0 0x7fd21c102f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:34.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd221fc8700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd21c108850 0x7fd21c101ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:34.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd220fc6700 1 -- 172.21.15.150:0/2831429459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd21c1035f0 msgr2=0x7fd21c102f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:34.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd220fc6700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd21c1035f0 0x7fd21c102f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:34.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd220fc6700 1 -- 172.21.15.150:0/2831429459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd21c108850 msgr2=0x7fd21c101ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:34.911 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd220fc6700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd21c108850 0x7fd21c101ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:34.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd220fc6700 1 -- 172.21.15.150:0/2831429459 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c102390 con 0x7fd21c105f20 2024-04-25T11:59:34.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.913+0000 7fd221fc8700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd21c108850 0x7fd21c101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:59:34.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.914+0000 7fd220fc6700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd21c105f20 0x7fd21c1015a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00d980 tx=0x7fd20c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:59:34.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.914+0000 7fd2127fc700 1 -- 172.21.15.150:0/2831429459 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c016070 con 0x7fd21c105f20 2024-04-25T11:59:34.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.914+0000 7fd2127fc700 1 -- 172.21.15.150:0/2831429459 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd20c004d00 con 0x7fd21c105f20 2024-04-25T11:59:34.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.914+0000 7fd223a2b700 1 -- 172.21.15.150:0/2831429459 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21c1a2e20 con 0x7fd21c105f20 2024-04-25T11:59:34.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.914+0000 7fd2127fc700 1 -- 172.21.15.150:0/2831429459 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c005610 con 0x7fd21c105f20 2024-04-25T11:59:34.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.914+0000 7fd223a2b700 1 -- 172.21.15.150:0/2831429459 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21c1a3280 con 0x7fd21c105f20 2024-04-25T11:59:34.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.915+0000 7fd223a2b700 1 -- 172.21.15.150:0/2831429459 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd21c069730 con 0x7fd21c105f20 2024-04-25T11:59:34.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.916+0000 7fd2127fc700 1 -- 172.21.15.150:0/2831429459 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fd20c004750 con 0x7fd21c105f20 2024-04-25T11:59:34.912 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.916+0000 7fd2127fc700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd2080a3bb0 0x7fd2080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:34.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.917+0000 7fd2127fc700 1 -- 172.21.15.150:0/2831429459 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fd20c0ce6e0 con 0x7fd21c105f20 2024-04-25T11:59:34.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.917+0000 7fd2217c7700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd2080a3bb0 0x7fd2080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:34.913 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.917+0000 7fd2217c7700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd2080a3bb0 0x7fd2080a6070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd218003f90 tx=0x7fd218009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:59:34.915 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:34.921+0000 7fd2127fc700 1 -- 172.21.15.150:0/2831429459 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd20c01f020 con 0x7fd21c105f20 2024-04-25T11:59:35.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:34 smithi019 ceph-mon[29213]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:35.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:34 smithi019 ceph-mon[29213]: Detected new or changed devices on smithi019 2024-04-25T11:59:35.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:34 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi019 to 17093M 2024-04-25T11:59:35.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:34 smithi019 ceph-mon[29213]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T11:59:35.081 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:34 smithi169 ceph-mon[32710]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:59:35.081 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:34 smithi169 ceph-mon[32710]: Detected new or changed devices on smithi019 2024-04-25T11:59:35.081 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:34 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi019 to 17093M 2024-04-25T11:59:35.081 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:34 smithi169 ceph-mon[32710]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T11:59:35.121 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:35.127+0000 7fd223a2b700 1 -- 172.21.15.150:0/2831429459 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd21c062e80 con 0x7fd2080a3bb0 2024-04-25T11:59:35.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:35 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:59:35.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:35 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:59:35.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:35 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:36.011 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:35 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:59:36.011 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:35 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:59:36.011 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:35 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:36.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:59:36.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:59:36.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:36.944 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:36 smithi150 ceph-mon[32250]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:36.944 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:36 smithi150 ceph-mon[32250]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:59:37.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:36 smithi019 ceph-mon[29213]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:37.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:36 smithi019 ceph-mon[29213]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:59:37.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:36 smithi169 ceph-mon[32710]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:37.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:36 smithi169 ceph-mon[32710]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:59:38.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:38 smithi019 ceph-mon[29213]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:38.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:38 smithi019 ceph-mon[29213]: from='client.? 172.21.15.150:0/2559425530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0"}]: dispatch 2024-04-25T11:59:38.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:38 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0"}]: dispatch 2024-04-25T11:59:38.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:38 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0"}]': finished 2024-04-25T11:59:38.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:38 smithi019 ceph-mon[29213]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T11:59:38.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:38.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:38 smithi019 ceph-mon[29213]: from='client.? 172.21.15.150:0/1630001426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:59:39.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:38 smithi169 ceph-mon[32710]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:39.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:38 smithi169 ceph-mon[32710]: from='client.? 172.21.15.150:0/2559425530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0"}]: dispatch 2024-04-25T11:59:39.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:38 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0"}]: dispatch 2024-04-25T11:59:39.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:38 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0"}]': finished 2024-04-25T11:59:39.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:38 smithi169 ceph-mon[32710]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T11:59:39.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:39.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:38 smithi169 ceph-mon[32710]: from='client.? 172.21.15.150:0/1630001426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:59:39.159 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:38 smithi150 ceph-mon[32250]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:39.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:38 smithi150 ceph-mon[32250]: from='client.? 172.21.15.150:0/2559425530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0"}]: dispatch 2024-04-25T11:59:39.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:38 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0"}]: dispatch 2024-04-25T11:59:39.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:38 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0"}]': finished 2024-04-25T11:59:39.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:38 smithi150 ceph-mon[32250]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T11:59:39.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:39.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:38 smithi150 ceph-mon[32250]: from='client.? 172.21.15.150:0/1630001426' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:59:41.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:40 smithi019 ceph-mon[29213]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:41.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:40 smithi169 ceph-mon[32710]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:41.159 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:40 smithi150 ceph-mon[32250]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:42.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:42 smithi150 ceph-mon[32250]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:43.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:42 smithi019 ceph-mon[29213]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:43.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:42 smithi169 ceph-mon[32710]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:44.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:44 smithi150 ceph-mon[32250]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:45.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:44 smithi019 ceph-mon[29213]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:45.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:44 smithi169 ceph-mon[32710]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:45.766 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:45 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T11:59:45.766 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:45 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:45.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T11:59:45.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:46.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T11:59:46.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:46.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:46 smithi150 ceph-mon[32250]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:46.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:46 smithi150 ceph-mon[32250]: Deploying daemon osd.1 on smithi150 2024-04-25T11:59:46.945 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:46 smithi019 ceph-mon[29213]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:46.945 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:46 smithi019 ceph-mon[29213]: Deploying daemon osd.1 on smithi150 2024-04-25T11:59:47.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:46 smithi169 ceph-mon[32710]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:47.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:46 smithi169 ceph-mon[32710]: Deploying daemon osd.1 on smithi150 2024-04-25T11:59:48.763 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:48 smithi150 ceph-mon[32250]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:49.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:48 smithi019 ceph-mon[29213]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:49.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:48 smithi169 ceph-mon[32710]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:50.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:50 smithi150 ceph-mon[32250]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:50.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:50 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:50.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:50 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:50.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:50 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:51.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:50 smithi019 ceph-mon[29213]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:51.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:50 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:51.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:50 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:51.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:50 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:51.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:50 smithi169 ceph-mon[32710]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:51.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:50 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:51.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:50 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:51.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:50 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:52.787 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:52 smithi150 ceph-mon[32250]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:53.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:52 smithi019 ceph-mon[29213]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:53.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:52 smithi169 ceph-mon[32710]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:54.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:54 smithi150 ceph-mon[32250]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:54.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:54.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:54.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:54.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:54.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:54.240 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:54.245+0000 7fd2127fc700 1 -- 172.21.15.150:0/2831429459 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd21c062e80 con 0x7fd2080a3bb0 2024-04-25T11:59:54.240 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 1 on host 'smithi150' 2024-04-25T11:59:54.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:54.247+0000 7fd223a2b700 1 -- 172.21.15.150:0/2831429459 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd2080a3bb0 msgr2=0x7fd2080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:54.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:54.247+0000 7fd223a2b700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd2080a3bb0 0x7fd2080a6070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd218003f90 tx=0x7fd218009180 comp rx=0 tx=0).stop 2024-04-25T11:59:54.242 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:54.247+0000 7fd223a2b700 1 -- 172.21.15.150:0/2831429459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd21c105f20 msgr2=0x7fd21c1015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:54.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:54.247+0000 7fd223a2b700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd21c105f20 0x7fd21c1015a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00d980 tx=0x7fd20c00dc90 comp rx=0 tx=0).stop 2024-04-25T11:59:54.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:54.247+0000 7fd223a2b700 1 -- 172.21.15.150:0/2831429459 shutdown_connections 2024-04-25T11:59:54.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:54.247+0000 7fd223a2b700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd2080a3bb0 0x7fd2080a6070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:54.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:54.248+0000 7fd223a2b700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd21c108850 0x7fd21c101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:54.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:54.248+0000 7fd223a2b700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd21c1035f0 0x7fd21c102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:54.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:54.248+0000 7fd223a2b700 1 --2- 172.21.15.150:0/2831429459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd21c105f20 0x7fd21c1015a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:54.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:54.248+0000 7fd223a2b700 1 -- 172.21.15.150:0/2831429459 >> 172.21.15.150:0/2831429459 conn(0x7fd21c0fcf40 msgr2=0x7fd21c103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:59:54.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:54.248+0000 7fd223a2b700 1 -- 172.21.15.150:0/2831429459 shutdown_connections 2024-04-25T11:59:54.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T11:59:54.248+0000 7fd223a2b700 1 -- 172.21.15.150:0/2831429459 wait complete. 2024-04-25T11:59:54.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:54 smithi169 ceph-mon[32710]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:54.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:54.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:54.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:54.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:54.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:54.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:54 smithi019 ceph-mon[29213]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:54.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:54.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:54.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:54.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:54.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:54.957 DEBUG:teuthology.orchestra.run.smithi150:osd.1> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T11:59:54.959 INFO:tasks.cephadm:Deploying osd.2 on smithi169 with /dev/vg_nvme/lv_4... 2024-04-25T11:59:54.960 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T11:59:55.208 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:59:55.331 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:55.331 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:55.331 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:55.332 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:55 smithi169 ceph-mon[32710]: from='osd.1 [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:59:55.332 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:55 smithi169 ceph-mon[32710]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:59:55.458 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:55.458 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:55.458 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:55.459 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:55 smithi150 ceph-mon[32250]: from='osd.1 [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:59:55.459 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:55 smithi150 ceph-mon[32250]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:59:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:59:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:55 smithi019 ceph-mon[29213]: from='osd.1 [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:59:55.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:55 smithi019 ceph-mon[29213]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:59:56.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:56 smithi169 ceph-mon[32710]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:56.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:56 smithi169 ceph-mon[32710]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T11:59:56.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:56 smithi169 ceph-mon[32710]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T11:59:56.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:56.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:56 smithi169 ceph-mon[32710]: from='osd.1 [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T11:59:56.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:56 smithi169 ceph-mon[32710]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T11:59:56.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:56 smithi019 ceph-mon[29213]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:56.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:56 smithi019 ceph-mon[29213]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T11:59:56.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:56 smithi019 ceph-mon[29213]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T11:59:56.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:56.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:56 smithi019 ceph-mon[29213]: from='osd.1 [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T11:59:56.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:56 smithi019 ceph-mon[29213]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T11:59:56.591 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:56 smithi150 ceph-mon[32250]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:56.591 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:56 smithi150 ceph-mon[32250]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T11:59:56.591 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:56 smithi150 ceph-mon[32250]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T11:59:56.591 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:56.591 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:56 smithi150 ceph-mon[32250]: from='osd.1 [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T11:59:56.591 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:56 smithi150 ceph-mon[32250]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T11:59:56.910 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 11:59:56 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-1[36869]: 2024-04-25T11:59:56.708+0000 7fe2f8dfd700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T11:59:56.910 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 11:59:56 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-1[36869]: 2024-04-25T11:59:56.712+0000 7fe2f0be8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:59:56.929 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T11:59:56.930 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T11:59:56.930 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T11:59:56.930 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T11:59:56.930 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115109 s, 911 MB/s 2024-04-25T11:59:56.930 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T11:59:57.526 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:57 smithi169 ceph-mon[32710]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T11:59:57.527 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:57 smithi169 ceph-mon[32710]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T11:59:57.527 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:57.527 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:57.527 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:57 smithi169 ceph-mon[32710]: from='osd.1 ' entity='osd.1' 2024-04-25T11:59:57.527 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:57.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:57 smithi019 ceph-mon[29213]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T11:59:57.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:57 smithi019 ceph-mon[29213]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T11:59:57.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:57.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:57.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:57 smithi019 ceph-mon[29213]: from='osd.1 ' entity='osd.1' 2024-04-25T11:59:57.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:57.555 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_4 2024-04-25T11:59:57.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:57 smithi150 ceph-mon[32250]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T11:59:57.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:57 smithi150 ceph-mon[32250]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T11:59:57.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:57.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:57.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:57 smithi150 ceph-mon[32250]: from='osd.1 ' entity='osd.1' 2024-04-25T11:59:57.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:57.800 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:59:58.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:58 smithi019 ceph-mon[29213]: purged_snaps scrub starts 2024-04-25T11:59:58.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:58 smithi019 ceph-mon[29213]: purged_snaps scrub ok 2024-04-25T11:59:58.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:58 smithi019 ceph-mon[29213]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:58.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:58 smithi019 ceph-mon[29213]: osd.1 [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601] boot 2024-04-25T11:59:58.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:58 smithi019 ceph-mon[29213]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T11:59:58.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:58.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:58.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:58.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:59:58.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:58.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:58.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:58.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:58 smithi169 ceph-mon[32710]: purged_snaps scrub starts 2024-04-25T11:59:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:58 smithi169 ceph-mon[32710]: purged_snaps scrub ok 2024-04-25T11:59:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:58 smithi169 ceph-mon[32710]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:58 smithi169 ceph-mon[32710]: osd.1 [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601] boot 2024-04-25T11:59:58.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:58 smithi169 ceph-mon[32710]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T11:59:58.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:58.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:58.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:58.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:59:58.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:58.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:58.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:58.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:58 smithi150 ceph-mon[32250]: purged_snaps scrub starts 2024-04-25T11:59:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:58 smithi150 ceph-mon[32250]: purged_snaps scrub ok 2024-04-25T11:59:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:58 smithi150 ceph-mon[32250]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:59:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:58 smithi150 ceph-mon[32250]: osd.1 [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601] boot 2024-04-25T11:59:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:58 smithi150 ceph-mon[32250]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T11:59:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:59:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:59:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:59:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:59:58.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T11:59:59.311 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:59 smithi169 ceph-mon[32710]: Detected new or changed devices on smithi150 2024-04-25T11:59:59.311 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 11:59:59 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi150 to 17088M 2024-04-25T11:59:59.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.419+0000 7f26ef521700 1 -- 172.21.15.169:0/2659677488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8104670 msgr2=0x7f26e810af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:59.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.419+0000 7f26ef521700 1 --2- 172.21.15.169:0/2659677488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8104670 0x7f26e810af20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009990 tx=0x7f26e0009ca0 comp rx=0 tx=0).stop 2024-04-25T11:59:59.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.419+0000 7f26ef521700 1 -- 172.21.15.169:0/2659677488 shutdown_connections 2024-04-25T11:59:59.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.419+0000 7f26ef521700 1 --2- 172.21.15.169:0/2659677488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e8102ad0 0x7f26e8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:59.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.419+0000 7f26ef521700 1 --2- 172.21.15.169:0/2659677488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8104670 0x7f26e810af20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:59.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.419+0000 7f26ef521700 1 --2- 172.21.15.169:0/2659677488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e8103cd0 0x7f26e8104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:59.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.419+0000 7f26ef521700 1 -- 172.21.15.169:0/2659677488 >> 172.21.15.169:0/2659677488 conn(0x7f26e80fe090 msgr2=0x7f26e81004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:59:59.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.419+0000 7f26ef521700 1 -- 172.21.15.169:0/2659677488 shutdown_connections 2024-04-25T11:59:59.416 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.419+0000 7f26ef521700 1 -- 172.21.15.169:0/2659677488 wait complete. 2024-04-25T11:59:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ef521700 1 Processor -- start 2024-04-25T11:59:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ef521700 1 -- start start 2024-04-25T11:59:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ef521700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8102ad0 0x7f26e80767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ef521700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e8103cd0 0x7f26e8074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ef521700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e8104670 0x7f26e8075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ef521700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26e810e700 con 0x7f26e8104670 2024-04-25T11:59:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ef521700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f26e810e580 con 0x7f26e8103cd0 2024-04-25T11:59:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ef521700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f26e810e880 con 0x7f26e8102ad0 2024-04-25T11:59:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ed2bd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8102ad0 0x7f26e80767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ed2bd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8102ad0 0x7f26e80767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:41540/0 (socket says 172.21.15.169:41540) 2024-04-25T11:59:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ed2bd700 1 -- 172.21.15.169:0/4269843146 learned_addr learned my addr 172.21.15.169:0/4269843146 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T11:59:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ecabc700 1 --2- 172.21.15.169:0/4269843146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e8103cd0 0x7f26e8074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:59.417 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26edabe700 1 --2- 172.21.15.169:0/4269843146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e8104670 0x7f26e8075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:59.418 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ed2bd700 1 -- 172.21.15.169:0/4269843146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e8103cd0 msgr2=0x7f26e8074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:59.418 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ed2bd700 1 --2- 172.21.15.169:0/4269843146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e8103cd0 0x7f26e8074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:59.418 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ed2bd700 1 -- 172.21.15.169:0/4269843146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e8104670 msgr2=0x7f26e8075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:59:59.418 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ed2bd700 1 --2- 172.21.15.169:0/4269843146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e8104670 0x7f26e8075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:59:59.418 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ed2bd700 1 -- 172.21.15.169:0/4269843146 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e8075c20 con 0x7f26e8102ad0 2024-04-25T11:59:59.418 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.420+0000 7f26ed2bd700 1 --2- 172.21.15.169:0/4269843146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8102ad0 0x7f26e80767e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f26e4007ee0 tx=0x7f26e400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:59:59.418 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.421+0000 7f26de7fc700 1 -- 172.21.15.169:0/4269843146 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e401a070 con 0x7f26e8102ad0 2024-04-25T11:59:59.418 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.421+0000 7f26de7fc700 1 -- 172.21.15.169:0/4269843146 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f26e40158d0 con 0x7f26e8102ad0 2024-04-25T11:59:59.418 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.421+0000 7f26ef521700 1 -- 172.21.15.169:0/4269843146 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e81a2e20 con 0x7f26e8102ad0 2024-04-25T11:59:59.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.421+0000 7f26de7fc700 1 -- 172.21.15.169:0/4269843146 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e40147d0 con 0x7f26e8102ad0 2024-04-25T11:59:59.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.421+0000 7f26ef521700 1 -- 172.21.15.169:0/4269843146 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e81a3270 con 0x7f26e8102ad0 2024-04-25T11:59:59.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.422+0000 7f26ef521700 1 -- 172.21.15.169:0/4269843146 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e8069730 con 0x7f26e8102ad0 2024-04-25T11:59:59.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.423+0000 7f26de7fc700 1 -- 172.21.15.169:0/4269843146 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f26e4015400 con 0x7f26e8102ad0 2024-04-25T11:59:59.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.423+0000 7f26de7fc700 1 --2- 172.21.15.169:0/4269843146 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f26d40a3a80 0x7f26d40a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:59:59.419 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.423+0000 7f26de7fc700 1 -- 172.21.15.169:0/4269843146 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f26e40ce400 con 0x7f26e8102ad0 2024-04-25T11:59:59.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.423+0000 7f26ecabc700 1 --2- 172.21.15.169:0/4269843146 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f26d40a3a80 0x7f26d40a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:59:59.420 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.424+0000 7f26ecabc700 1 --2- 172.21.15.169:0/4269843146 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f26d40a3a80 0x7f26d40a5f40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f26d8006fd0 tx=0x7f26d8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:59:59.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.428+0000 7f26de7fc700 1 -- 172.21.15.169:0/4269843146 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26e4010070 con 0x7f26e8102ad0 2024-04-25T11:59:59.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:59 smithi019 ceph-mon[29213]: Detected new or changed devices on smithi150 2024-04-25T11:59:59.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 11:59:59 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi150 to 17088M 2024-04-25T11:59:59.627 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T11:59:59.632+0000 7f26ef521700 1 -- 172.21.15.169:0/4269843146 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f26e802e3b0 con 0x7f26d40a3a80 2024-04-25T11:59:59.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:59 smithi150 ceph-mon[32250]: Detected new or changed devices on smithi150 2024-04-25T11:59:59.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 11:59:59 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi150 to 17088M 2024-04-25T12:00:00.515 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:00 smithi169 ceph-mon[32710]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:00.515 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:00 smithi169 ceph-mon[32710]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T12:00:00.515 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:00 smithi169 ceph-mon[32710]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:00:00.515 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:00:00.515 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:00:00.515 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:00.515 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:00 smithi169 ceph-mon[32710]: overall HEALTH_OK 2024-04-25T12:00:00.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:00 smithi019 ceph-mon[29213]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:00.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:00 smithi019 ceph-mon[29213]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T12:00:00.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:00 smithi019 ceph-mon[29213]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:00:00.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:00:00.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:00:00.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:00.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:00 smithi019 ceph-mon[29213]: overall HEALTH_OK 2024-04-25T12:00:00.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:00 smithi150 ceph-mon[32250]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:00.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:00 smithi150 ceph-mon[32250]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T12:00:00.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:00 smithi150 ceph-mon[32250]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:00:00.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:00:00.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:00:00.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:00.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:00 smithi150 ceph-mon[32250]: overall HEALTH_OK 2024-04-25T12:00:02.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:02 smithi019 ceph-mon[29213]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:02.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:02 smithi169 ceph-mon[32710]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:02.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:02 smithi150 ceph-mon[32250]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:03.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:03 smithi019 ceph-mon[29213]: from='client.? 172.21.15.169:0/1114940612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a12732f-3dd6-4d97-b62a-2bd66a797b1c"}]: dispatch 2024-04-25T12:00:03.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:03 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a12732f-3dd6-4d97-b62a-2bd66a797b1c"}]: dispatch 2024-04-25T12:00:03.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:03 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a12732f-3dd6-4d97-b62a-2bd66a797b1c"}]': finished 2024-04-25T12:00:03.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:03 smithi019 ceph-mon[29213]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T12:00:03.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:03 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:03.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:03 smithi019 ceph-mon[29213]: from='client.? 172.21.15.169:0/3960413634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:00:03.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:03 smithi169 ceph-mon[32710]: from='client.? 172.21.15.169:0/1114940612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a12732f-3dd6-4d97-b62a-2bd66a797b1c"}]: dispatch 2024-04-25T12:00:03.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:03 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a12732f-3dd6-4d97-b62a-2bd66a797b1c"}]: dispatch 2024-04-25T12:00:03.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:03 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a12732f-3dd6-4d97-b62a-2bd66a797b1c"}]': finished 2024-04-25T12:00:03.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:03 smithi169 ceph-mon[32710]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T12:00:03.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:03 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:03.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:03 smithi169 ceph-mon[32710]: from='client.? 172.21.15.169:0/3960413634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:00:03.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:03 smithi150 ceph-mon[32250]: from='client.? 172.21.15.169:0/1114940612' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a12732f-3dd6-4d97-b62a-2bd66a797b1c"}]: dispatch 2024-04-25T12:00:03.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:03 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a12732f-3dd6-4d97-b62a-2bd66a797b1c"}]: dispatch 2024-04-25T12:00:03.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:03 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a12732f-3dd6-4d97-b62a-2bd66a797b1c"}]': finished 2024-04-25T12:00:03.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:03 smithi150 ceph-mon[32250]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T12:00:03.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:03 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:03.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:03 smithi150 ceph-mon[32250]: from='client.? 172.21.15.169:0/3960413634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:00:04.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:04 smithi169 ceph-mon[32710]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:04.659 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:04 smithi150 ceph-mon[32250]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:04.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:04 smithi019 ceph-mon[29213]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:06.551 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:06 smithi169 ceph-mon[32710]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:06.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:06 smithi150 ceph-mon[32250]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:06.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:06 smithi019 ceph-mon[29213]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:08.526 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:08 smithi169 ceph-mon[32710]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:08.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:08 smithi150 ceph-mon[32250]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:08.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:08 smithi019 ceph-mon[29213]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:09.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:09 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T12:00:09.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:09 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:09.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:09 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T12:00:09.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:09 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:09.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:09 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T12:00:09.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:09 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:10.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:10 smithi169 ceph-mon[32710]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:10.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:10 smithi169 ceph-mon[32710]: Deploying daemon osd.2 on smithi169 2024-04-25T12:00:10.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:10 smithi150 ceph-mon[32250]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:10.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:10 smithi150 ceph-mon[32250]: Deploying daemon osd.2 on smithi169 2024-04-25T12:00:10.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:10 smithi019 ceph-mon[29213]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:10.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:10 smithi019 ceph-mon[29213]: Deploying daemon osd.2 on smithi169 2024-04-25T12:00:12.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:12 smithi169 ceph-mon[32710]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:12.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:12 smithi150 ceph-mon[32250]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:12.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:12 smithi019 ceph-mon[29213]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:14.402 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:14 smithi169 ceph-mon[32710]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:14.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:14 smithi150 ceph-mon[32250]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:14.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:14 smithi019 ceph-mon[29213]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:15.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:00:15.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:15.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:15.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:00:15.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:15.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:15.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:00:15.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:15.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:16.652 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:16 smithi169 ceph-mon[32710]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:16.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:16 smithi019 ceph-mon[29213]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:16.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:16 smithi150 ceph-mon[32250]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:17.746 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:00:17.751+0000 7f26de7fc700 1 -- 172.21.15.169:0/4269843146 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f26e802e3b0 con 0x7f26d40a3a80 2024-04-25T12:00:17.746 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 2 on host 'smithi169' 2024-04-25T12:00:17.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:00:17.753+0000 7f26ef521700 1 -- 172.21.15.169:0/4269843146 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f26d40a3a80 msgr2=0x7f26d40a5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:00:17.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:00:17.753+0000 7f26ef521700 1 --2- 172.21.15.169:0/4269843146 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f26d40a3a80 0x7f26d40a5f40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f26d8006fd0 tx=0x7f26d8008040 comp rx=0 tx=0).stop 2024-04-25T12:00:17.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:00:17.753+0000 7f26ef521700 1 -- 172.21.15.169:0/4269843146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8102ad0 msgr2=0x7f26e80767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:00:17.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:00:17.753+0000 7f26ef521700 1 --2- 172.21.15.169:0/4269843146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8102ad0 0x7f26e80767e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f26e4007ee0 tx=0x7f26e400d3b0 comp rx=0 tx=0).stop 2024-04-25T12:00:17.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:00:17.753+0000 7f26ef521700 1 -- 172.21.15.169:0/4269843146 shutdown_connections 2024-04-25T12:00:17.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:00:17.753+0000 7f26ef521700 1 --2- 172.21.15.169:0/4269843146 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f26d40a3a80 0x7f26d40a5f40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:17.748 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:00:17.753+0000 7f26ef521700 1 --2- 172.21.15.169:0/4269843146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e8104670 0x7f26e8075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:17.749 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:00:17.753+0000 7f26ef521700 1 --2- 172.21.15.169:0/4269843146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26e8102ad0 0x7f26e80767e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:17.749 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:00:17.753+0000 7f26ef521700 1 --2- 172.21.15.169:0/4269843146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26e8103cd0 0x7f26e8074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:17.749 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:00:17.753+0000 7f26ef521700 1 -- 172.21.15.169:0/4269843146 >> 172.21.15.169:0/4269843146 conn(0x7f26e80fe090 msgr2=0x7f26e8106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:00:17.749 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:00:17.753+0000 7f26ef521700 1 -- 172.21.15.169:0/4269843146 shutdown_connections 2024-04-25T12:00:17.749 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:00:17.753+0000 7f26ef521700 1 -- 172.21.15.169:0/4269843146 wait complete. 2024-04-25T12:00:18.305 DEBUG:teuthology.orchestra.run.smithi169:osd.2> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T12:00:18.307 INFO:tasks.cephadm:Deploying osd.3 on smithi019 with /dev/vg_nvme/lv_3... 2024-04-25T12:00:18.307 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T12:00:18.555 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:00:18.609 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:00:18 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2[38026]: 2024-04-25T12:00:18.371+0000 7f147e4af7c0 -1 osd.2 0 log_to_monitors true 2024-04-25T12:00:18.902 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:18 smithi169 ceph-mon[32710]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:18.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:18 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:18.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:18 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:18.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:18 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:18.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:18 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:18.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:18 smithi169 ceph-mon[32710]: from='osd.2 [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:00:18.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:18 smithi169 ceph-mon[32710]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:00:19.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:18 smithi019 ceph-mon[29213]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:19.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:19.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:19.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:19.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:19.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:18 smithi019 ceph-mon[29213]: from='osd.2 [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:00:19.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:18 smithi019 ceph-mon[29213]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:00:19.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:18 smithi150 ceph-mon[32250]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:19.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:18 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:19.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:18 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:19.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:18 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:19.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:18 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:19.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:18 smithi150 ceph-mon[32250]: from='osd.2 [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:00:19.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:18 smithi150 ceph-mon[32250]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T12:00:19.902 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:00:19 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2[38026]: 2024-04-25T12:00:19.807+0000 7f1473b14700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T12:00:19.903 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:00:19 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2[38026]: 2024-04-25T12:00:19.811+0000 7f146f907700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:00:19.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:19 smithi169 ceph-mon[32710]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T12:00:19.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:19 smithi169 ceph-mon[32710]: from='osd.2 [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:00:19.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:19 smithi169 ceph-mon[32710]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T12:00:19.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:19 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:19.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:19 smithi169 ceph-mon[32710]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:00:19.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:19 smithi169 ceph-mon[32710]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:19.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:19 smithi019 ceph-mon[29213]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T12:00:19.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:19 smithi019 ceph-mon[29213]: from='osd.2 [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:00:19.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:19 smithi019 ceph-mon[29213]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T12:00:19.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:19 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:19.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:19 smithi019 ceph-mon[29213]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:00:19.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:19 smithi019 ceph-mon[29213]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:20.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:19 smithi150 ceph-mon[32250]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T12:00:20.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:19 smithi150 ceph-mon[32250]: from='osd.2 [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:00:20.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:19 smithi150 ceph-mon[32250]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T12:00:20.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:19 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:20.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:19 smithi150 ceph-mon[32250]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:00:20.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:19 smithi150 ceph-mon[32250]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T12:00:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T12:00:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T12:00:20.519 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-25T12:00:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-25T12:00:20.519 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113883 s, 921 MB/s 2024-04-25T12:00:20.519 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-25T12:00:21.011 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:20 smithi169 ceph-mon[32710]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T12:00:21.011 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:20 smithi169 ceph-mon[32710]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T12:00:21.011 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:20 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:21.011 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:20 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:21.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:20 smithi150 ceph-mon[32250]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T12:00:21.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:20 smithi150 ceph-mon[32250]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T12:00:21.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:20 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:21.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:20 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:21.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:20 smithi019 ceph-mon[29213]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T12:00:21.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:20 smithi019 ceph-mon[29213]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T12:00:21.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:20 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:21.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:20 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:21.195 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_3 2024-04-25T12:00:21.445 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:00:21.826 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: purged_snaps scrub starts 2024-04-25T12:00:21.827 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: purged_snaps scrub ok 2024-04-25T12:00:21.827 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: osd.2 [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528] boot 2024-04-25T12:00:21.827 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T12:00:21.827 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:21.827 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:21.827 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:00:21.827 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: Detected new or changed devices on smithi169 2024-04-25T12:00:21.827 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:21.827 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:21.827 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:00:21.827 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-25T12:00:21.828 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:21.828 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:21.828 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:00:21.828 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:21.828 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:00:21.828 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:21.828 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:00:21.828 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:22.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: purged_snaps scrub starts 2024-04-25T12:00:22.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: purged_snaps scrub ok 2024-04-25T12:00:22.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: osd.2 [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528] boot 2024-04-25T12:00:22.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T12:00:22.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:22.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:22.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:00:22.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: Detected new or changed devices on smithi169 2024-04-25T12:00:22.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:22.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:22.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:00:22.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-25T12:00:22.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:22.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:22.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:00:22.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:22.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:00:22.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:22.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:00:22.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:22.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: purged_snaps scrub starts 2024-04-25T12:00:22.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: purged_snaps scrub ok 2024-04-25T12:00:22.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: osd.2 [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528] boot 2024-04-25T12:00:22.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T12:00:22.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:00:22.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:22.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:00:22.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: Detected new or changed devices on smithi169 2024-04-25T12:00:22.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:22.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:22.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:00:22.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi169 to 17088M 2024-04-25T12:00:22.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:22.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:22.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:00:22.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:22.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:00:22.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:22.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:00:22.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:22.869 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T12:00:22.869 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:22 smithi019 ceph-mon[29213]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T12:00:22.869 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:00:23.092 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T12:00:23.092 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:22 smithi169 ceph-mon[32710]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T12:00:23.092 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:00:23.143 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:00:22 smithi019 sudo[48697]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:00:23.144 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:00:22 smithi019 sudo[48697]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:00:23.144 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:00:22 smithi019 sudo[48697]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:00:23.144 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:00:22 smithi019 sudo[48697]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:00:23.144 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:00:22 smithi019 sudo[48700]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:00:23.144 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:00:22 smithi019 sudo[48700]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:00:23.144 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:00:22 smithi019 sudo[48700]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:00:23.144 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:00:22 smithi019 sudo[48700]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:00:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.155+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/2529727092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca01019a0 msgr2=0x7f3ca0101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:00:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.155+0000 7f3ca7d7b700 1 --2- 172.21.15.19:0/2529727092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca01019a0 0x7f3ca0101dc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0098f0 tx=0x7f3c9c009c00 comp rx=0 tx=0).stop 2024-04-25T12:00:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.155+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/2529727092 shutdown_connections 2024-04-25T12:00:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.155+0000 7f3ca7d7b700 1 --2- 172.21.15.19:0/2529727092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca0103540 0x7f3ca0109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.155+0000 7f3ca7d7b700 1 --2- 172.21.15.19:0/2529727092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca01019a0 0x7f3ca0101dc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.155+0000 7f3ca7d7b700 1 --2- 172.21.15.19:0/2529727092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ca0102ba0 0x7f3ca0103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.155+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/2529727092 >> 172.21.15.19:0/2529727092 conn(0x7f3ca00fcf40 msgr2=0x7f3ca00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:00:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.155+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/2529727092 shutdown_connections 2024-04-25T12:00:23.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.155+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/2529727092 wait complete. 2024-04-25T12:00:23.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca7d7b700 1 Processor -- start 2024-04-25T12:00:23.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca7d7b700 1 -- start start 2024-04-25T12:00:23.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca7d7b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca01019a0 0x7f3ca019af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:00:23.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca7d7b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ca0102ba0 0x7f3ca019b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:00:23.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca7d7b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca0103540 0x7f3ca01a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:00:23.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca7d7b700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca010d550 con 0x7f3ca01019a0 2024-04-25T12:00:23.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca7d7b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca010d3d0 con 0x7f3ca0102ba0 2024-04-25T12:00:23.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca7d7b700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca010d6d0 con 0x7f3ca0103540 2024-04-25T12:00:23.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca5b17700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca01019a0 0x7f3ca019af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:00:23.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca5b17700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca01019a0 0x7f3ca019af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33526/0 (socket says 172.21.15.19:33526) 2024-04-25T12:00:23.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca5b17700 1 -- 172.21.15.19:0/797407690 learned_addr learned my addr 172.21.15.19:0/797407690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:00:23.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca5316700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ca0102ba0 0x7f3ca019b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:00:23.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca6318700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca0103540 0x7f3ca01a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:00:23.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca6318700 1 -- 172.21.15.19:0/797407690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ca0102ba0 msgr2=0x7f3ca019b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:00:23.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca6318700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ca0102ba0 0x7f3ca019b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:23.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca6318700 1 -- 172.21.15.19:0/797407690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca01019a0 msgr2=0x7f3ca019af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:00:23.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca6318700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca01019a0 0x7f3ca019af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:23.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca6318700 1 -- 172.21.15.19:0/797407690 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca01a2bf0 con 0x7f3ca0103540 2024-04-25T12:00:23.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.156+0000 7f3ca5b17700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca01019a0 0x7f3ca019af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:00:23.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.157+0000 7f3ca6318700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca0103540 0x7f3ca01a2550 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009f60 tx=0x7f3c98009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:00:23.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.157+0000 7f3c96ffd700 1 -- 172.21.15.19:0/797407690 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c980092e0 con 0x7f3ca0103540 2024-04-25T12:00:23.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.157+0000 7f3c96ffd700 1 -- 172.21.15.19:0/797407690 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3c98004bb0 con 0x7f3ca0103540 2024-04-25T12:00:23.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.157+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/797407690 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca01a2e80 con 0x7f3ca0103540 2024-04-25T12:00:23.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.157+0000 7f3c96ffd700 1 -- 172.21.15.19:0/797407690 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98004020 con 0x7f3ca0103540 2024-04-25T12:00:23.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.157+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/797407690 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca01a3330 con 0x7f3ca0103540 2024-04-25T12:00:23.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.158+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/797407690 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca0069730 con 0x7f3ca0103540 2024-04-25T12:00:23.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.159+0000 7f3c96ffd700 1 -- 172.21.15.19:0/797407690 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f3c9801b030 con 0x7f3ca0103540 2024-04-25T12:00:23.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.159+0000 7f3c96ffd700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3c8c0a3bb0 0x7f3c8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:00:23.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.159+0000 7f3ca5b17700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3c8c0a3bb0 0x7f3c8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:00:23.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.160+0000 7f3c96ffd700 1 -- 172.21.15.19:0/797407690 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f3c980ce950 con 0x7f3ca0103540 2024-04-25T12:00:23.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.160+0000 7f3ca5b17700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3c8c0a3bb0 0x7f3c8c0a6070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009680 tx=0x7f3c9c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:00:23.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.164+0000 7f3c96ffd700 1 -- 172.21.15.19:0/797407690 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c98022020 con 0x7f3ca0103540 2024-04-25T12:00:23.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T12:00:23.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:22 smithi150 ceph-mon[32250]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T12:00:23.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:00:23.160 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41011]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:00:23.160 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41011]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:00:23.160 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41011]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:00:23.160 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41011]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:00:23.160 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41014]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:00:23.160 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41014]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:00:23.160 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41014]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:00:23.161 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41014]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:00:23.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:23.395+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/797407690 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3ca006ae60 con 0x7f3c8c0a3bb0 2024-04-25T12:00:23.402 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42084]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:00:23.403 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42084]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:00:23.403 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42084]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:00:23.403 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42084]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:00:23.403 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42087]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:00:23.403 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42087]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:00:23.403 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42087]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:00:23.403 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42087]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:00:23.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42090]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:00:23.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42090]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:00:23.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42090]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:00:23.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42090]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:00:23.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42093]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:00:23.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42093]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:00:23.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42093]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:00:23.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 sudo[42093]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:00:23.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 sudo[48717]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:00:23.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 sudo[48717]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:00:23.544 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 sudo[48717]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:00:23.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 sudo[48717]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:00:23.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 sudo[48729]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:00:23.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 sudo[48729]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:00:23.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 sudo[48729]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:00:23.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 sudo[48729]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:00:23.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41017]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T12:00:23.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41017]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:00:23.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41017]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:00:23.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41017]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:00:23.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41020]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T12:00:23.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41020]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T12:00:23.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41020]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T12:00:23.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 sudo[41020]: pam_unix(sudo:session): session closed for user root 2024-04-25T12:00:24.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T12:00:24.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T12:00:24.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:24.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:24.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:24.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:24.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:00:24.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:00:24.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:00:24.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:00:24.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:00:24.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:00:24.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:00:24.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:00:24.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:00:24.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:00:24.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:00:24.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:00:24.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:00:24.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:00:24.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:00:24.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:00:24.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:00:24.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:00:24.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:24.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T12:00:24.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T12:00:24.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:24.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:24.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:24.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:24.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:00:24.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:00:24.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:00:24.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:00:24.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:00:24.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:00:24.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:00:24.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:00:24.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:00:24.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:00:24.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:00:24.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:00:24.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:00:24.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:00:24.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:00:24.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:00:24.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:00:24.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:00:24.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:24.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T12:00:24.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T12:00:24.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:24.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:24.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:24.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:24.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:00:24.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:00:24.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:00:24.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:00:24.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:00:24.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:00:24.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:00:24.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:00:24.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:00:24.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:00:24.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T12:00:24.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:00:24.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:00:24.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:00:24.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T12:00:24.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:00:24.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:00:24.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:00:24.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:25.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:24 smithi169 ceph-mon[32710]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T12:00:25.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:24 smithi150 ceph-mon[32250]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T12:00:25.163 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:24 smithi019 ceph-mon[29213]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T12:00:25.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:25.844+0000 7f3c96ffd700 1 -- 172.21.15.19:0/797407690 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3c98012050 con 0x7f3ca0103540 2024-04-25T12:00:26.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:25 smithi169 ceph-mon[32710]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:26.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:25 smithi150 ceph-mon[32250]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:26.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:25 smithi019 ceph-mon[29213]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:27.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:26 smithi169 ceph-mon[32710]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T12:00:27.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:26 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2011928990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "518328cd-b7de-4a49-9045-2ebf6b0eb4ab"}]: dispatch 2024-04-25T12:00:27.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:26 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2011928990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "518328cd-b7de-4a49-9045-2ebf6b0eb4ab"}]': finished 2024-04-25T12:00:27.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:26 smithi169 ceph-mon[32710]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T12:00:27.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:27.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:26 smithi150 ceph-mon[32250]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T12:00:27.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:26 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2011928990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "518328cd-b7de-4a49-9045-2ebf6b0eb4ab"}]: dispatch 2024-04-25T12:00:27.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:26 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2011928990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "518328cd-b7de-4a49-9045-2ebf6b0eb4ab"}]': finished 2024-04-25T12:00:27.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:26 smithi150 ceph-mon[32250]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T12:00:27.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:27.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:26 smithi019 ceph-mon[29213]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T12:00:27.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:26 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2011928990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "518328cd-b7de-4a49-9045-2ebf6b0eb4ab"}]: dispatch 2024-04-25T12:00:27.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:26 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2011928990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "518328cd-b7de-4a49-9045-2ebf6b0eb4ab"}]': finished 2024-04-25T12:00:27.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:26 smithi019 ceph-mon[29213]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T12:00:27.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:28.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:27 smithi169 ceph-mon[32710]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:28.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:27 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/4144925692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:00:28.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:27 smithi150 ceph-mon[32250]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:28.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:27 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/4144925692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:00:28.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:27 smithi019 ceph-mon[29213]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:28.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:27 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/4144925692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:00:30.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:29 smithi169 ceph-mon[32710]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 52 KiB/s wr, 4 op/s 2024-04-25T12:00:30.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:29 smithi150 ceph-mon[32250]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 52 KiB/s wr, 4 op/s 2024-04-25T12:00:30.264 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:29 smithi019 ceph-mon[29213]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 52 KiB/s wr, 4 op/s 2024-04-25T12:00:32.044 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:31 smithi019 ceph-mon[29213]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 37 KiB/s wr, 3 op/s 2024-04-25T12:00:32.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:31 smithi169 ceph-mon[32710]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 37 KiB/s wr, 3 op/s 2024-04-25T12:00:32.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:31 smithi150 ceph-mon[32250]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 37 KiB/s wr, 3 op/s 2024-04-25T12:00:34.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:33 smithi169 ceph-mon[32710]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 29 KiB/s wr, 2 op/s 2024-04-25T12:00:34.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:33 smithi150 ceph-mon[32250]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 29 KiB/s wr, 2 op/s 2024-04-25T12:00:34.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:33 smithi019 ceph-mon[29213]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 29 KiB/s wr, 2 op/s 2024-04-25T12:00:35.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:34 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T12:00:35.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:34 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:35.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:34 smithi169 ceph-mon[32710]: Deploying daemon osd.3 on smithi019 2024-04-25T12:00:35.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:34 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T12:00:35.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:34 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:35.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:34 smithi150 ceph-mon[32250]: Deploying daemon osd.3 on smithi019 2024-04-25T12:00:35.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:34 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T12:00:35.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:34 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:35.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:34 smithi019 ceph-mon[29213]: Deploying daemon osd.3 on smithi019 2024-04-25T12:00:36.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:35 smithi169 ceph-mon[32710]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T12:00:36.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:35 smithi150 ceph-mon[32250]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T12:00:36.215 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:35 smithi019 ceph-mon[29213]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T12:00:38.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:37 smithi169 ceph-mon[32710]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 960 B/s rd, 25 KiB/s wr, 2 op/s 2024-04-25T12:00:38.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:37 smithi150 ceph-mon[32250]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 960 B/s rd, 25 KiB/s wr, 2 op/s 2024-04-25T12:00:38.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:37 smithi019 ceph-mon[29213]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 960 B/s rd, 25 KiB/s wr, 2 op/s 2024-04-25T12:00:40.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:40 smithi169 ceph-mon[32710]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-04-25T12:00:40.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:40.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:40.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:00:40.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:40 smithi150 ceph-mon[32250]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-04-25T12:00:40.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:40 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:40.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:40 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:40.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:40 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:00:40.936 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:40 smithi019 ceph-mon[29213]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-04-25T12:00:40.937 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:40 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:40.937 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:40 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:40.937 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:40 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:00:42.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:42 smithi019 ceph-mon[29213]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:42.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:42 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:42.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:42 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:42.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:42 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:42.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:42 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:00:42.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:42 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:42.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:42 smithi169 ceph-mon[32710]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:42.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:42 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:42.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:42 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:42.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:42 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:42.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:42 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:00:42.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:42 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:42.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:42 smithi150 ceph-mon[32250]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:42.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:42 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:42.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:42 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:42.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:42 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:42.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:42 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:00:42.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:42 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:43.450 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 3 on host 'smithi019' 2024-04-25T12:00:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:43.455+0000 7f3c96ffd700 1 -- 172.21.15.19:0/797407690 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3ca006ae60 con 0x7f3c8c0a3bb0 2024-04-25T12:00:43.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:43.458+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/797407690 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3c8c0a3bb0 msgr2=0x7f3c8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:00:43.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:43.458+0000 7f3ca7d7b700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3c8c0a3bb0 0x7f3c8c0a6070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009680 tx=0x7f3c9c019040 comp rx=0 tx=0).stop 2024-04-25T12:00:43.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:43.458+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/797407690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca0103540 msgr2=0x7f3ca01a2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:00:43.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:43.458+0000 7f3ca7d7b700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca0103540 0x7f3ca01a2550 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009f60 tx=0x7f3c98009f90 comp rx=0 tx=0).stop 2024-04-25T12:00:43.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:43.458+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/797407690 shutdown_connections 2024-04-25T12:00:43.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:43.458+0000 7f3ca7d7b700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3c8c0a3bb0 0x7f3c8c0a6070 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:43.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:43.458+0000 7f3ca7d7b700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca01019a0 0x7f3ca019af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:43.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:43.458+0000 7f3ca7d7b700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca0103540 0x7f3ca01a2550 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:43.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:43.458+0000 7f3ca7d7b700 1 --2- 172.21.15.19:0/797407690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ca0102ba0 0x7f3ca019b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:43.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:43.458+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/797407690 >> 172.21.15.19:0/797407690 conn(0x7f3ca00fcf40 msgr2=0x7f3ca006a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:00:43.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:43.458+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/797407690 shutdown_connections 2024-04-25T12:00:43.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:00:43.458+0000 7f3ca7d7b700 1 -- 172.21.15.19:0/797407690 wait complete. 2024-04-25T12:00:44.097 DEBUG:teuthology.orchestra.run.smithi019:osd.3> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T12:00:44.100 INFO:tasks.cephadm:Deploying osd.4 on smithi150 with /dev/vg_nvme/lv_3... 2024-04-25T12:00:44.100 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T12:00:44.343 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:00:44.708 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:44 smithi169 ceph-mon[32710]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:44.708 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:44 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:44.708 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:44 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:44.708 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:44 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:00:44.708 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:44 smithi169 ceph-mon[32710]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T12:00:44.715 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:44 smithi150 ceph-mon[32250]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:44.715 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:44 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:44.715 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:44 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:44.715 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:44 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:00:44.716 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:44 smithi150 ceph-mon[32250]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T12:00:44.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:44 smithi019 ceph-mon[29213]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:44.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:44 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:44.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:44 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:44.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:44 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:00:44.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:44 smithi019 ceph-mon[29213]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T12:00:45.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:45 smithi019 ceph-mon[29213]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T12:00:45.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:45 smithi019 ceph-mon[29213]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T12:00:45.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:45 smithi019 ceph-mon[29213]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T12:00:45.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:45.760 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:45 smithi169 ceph-mon[32710]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T12:00:45.760 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:45 smithi169 ceph-mon[32710]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T12:00:45.760 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:45 smithi169 ceph-mon[32710]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T12:00:45.760 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:45.787 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:45 smithi150 ceph-mon[32250]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T12:00:45.787 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:45 smithi150 ceph-mon[32250]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T12:00:45.787 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:45 smithi150 ceph-mon[32250]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T12:00:45.787 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:45 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:46.295 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:00:45 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-3[50980]: 2024-04-25T12:00:45.893+0000 7f333221d700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T12:00:46.295 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:00:45 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-3[50980]: 2024-04-25T12:00:45.899+0000 7f332a008700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:00:46.400 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T12:00:46.401 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T12:00:46.401 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-25T12:00:46.401 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-25T12:00:46.401 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011465 s, 915 MB/s 2024-04-25T12:00:46.401 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-25T12:00:46.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:46 smithi019 ceph-mon[29213]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:46.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:46 smithi019 ceph-mon[29213]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T12:00:46.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:46 smithi019 ceph-mon[29213]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T12:00:46.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:46 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:46.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:46 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:46.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:46 smithi019 ceph-mon[29213]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' 2024-04-25T12:00:46.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:46 smithi019 ceph-mon[29213]: osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913] boot 2024-04-25T12:00:46.579 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:46 smithi019 ceph-mon[29213]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T12:00:46.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:46 smithi169 ceph-mon[32710]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:46.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:46 smithi169 ceph-mon[32710]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T12:00:46.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:46 smithi169 ceph-mon[32710]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T12:00:46.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:46 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:46.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:46 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:46.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:46 smithi169 ceph-mon[32710]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' 2024-04-25T12:00:46.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:46 smithi169 ceph-mon[32710]: osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913] boot 2024-04-25T12:00:46.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:46 smithi169 ceph-mon[32710]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T12:00:46.749 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:46 smithi150 ceph-mon[32250]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T12:00:46.749 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:46 smithi150 ceph-mon[32250]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T12:00:46.749 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:46 smithi150 ceph-mon[32250]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T12:00:46.749 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:46 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:46.749 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:46 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:46.749 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:46 smithi150 ceph-mon[32250]: from='osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913]' entity='osd.3' 2024-04-25T12:00:46.749 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:46 smithi150 ceph-mon[32250]: osd.3 [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913] boot 2024-04-25T12:00:46.749 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:46 smithi150 ceph-mon[32250]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T12:00:47.101 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2024-04-25T12:00:47.347 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:00:47.567 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:47 smithi019 ceph-mon[29213]: purged_snaps scrub starts 2024-04-25T12:00:47.567 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:47 smithi019 ceph-mon[29213]: purged_snaps scrub ok 2024-04-25T12:00:47.567 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:47.567 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:47.567 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:47.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:00:47.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:00:47.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:47.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:47.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:00:47.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:47.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:47 smithi019 ceph-mon[29213]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T12:00:47.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:47 smithi150 ceph-mon[32250]: purged_snaps scrub starts 2024-04-25T12:00:47.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:47 smithi150 ceph-mon[32250]: purged_snaps scrub ok 2024-04-25T12:00:47.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:47.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:47.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:47.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:00:47.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:00:47.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:47.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:47.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:00:47.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:47.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:47 smithi150 ceph-mon[32250]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T12:00:47.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:47 smithi169 ceph-mon[32710]: purged_snaps scrub starts 2024-04-25T12:00:47.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:47 smithi169 ceph-mon[32710]: purged_snaps scrub ok 2024-04-25T12:00:47.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:00:47.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:47.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:47.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:00:47.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:00:47.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:47.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:47.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:00:47.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:00:47.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:47 smithi169 ceph-mon[32710]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T12:00:48.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:48 smithi150 ceph-mon[32250]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 130 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:00:48.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:48 smithi150 ceph-mon[32250]: Detected new or changed devices on smithi019 2024-04-25T12:00:48.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:48 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi019 to 8546M 2024-04-25T12:00:48.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:48 smithi150 ceph-mon[32250]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T12:00:48.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:48 smithi019 ceph-mon[29213]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 130 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:00:48.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:48 smithi019 ceph-mon[29213]: Detected new or changed devices on smithi019 2024-04-25T12:00:48.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:48 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi019 to 8546M 2024-04-25T12:00:48.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:48 smithi019 ceph-mon[29213]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T12:00:48.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:48 smithi169 ceph-mon[32710]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 130 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:00:48.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:48 smithi169 ceph-mon[32710]: Detected new or changed devices on smithi019 2024-04-25T12:00:48.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:48 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi019 to 8546M 2024-04-25T12:00:48.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:48 smithi169 ceph-mon[32710]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T12:00:48.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.966+0000 7f4aa1257700 1 -- 172.21.15.150:0/3281244888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a9c1035d0 msgr2=0x7f4a9c1059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:00:48.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.966+0000 7f4aa1257700 1 --2- 172.21.15.150:0/3281244888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a9c1035d0 0x7f4a9c1059c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009990 tx=0x7f4a8c009ca0 comp rx=0 tx=0).stop 2024-04-25T12:00:48.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.966+0000 7f4aa1257700 1 -- 172.21.15.150:0/3281244888 shutdown_connections 2024-04-25T12:00:48.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.966+0000 7f4aa1257700 1 --2- 172.21.15.150:0/3281244888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c108830 0x7f4a9c10ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:48.967 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.966+0000 7f4aa1257700 1 --2- 172.21.15.150:0/3281244888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a9c1035d0 0x7f4a9c1059c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:48.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.966+0000 7f4aa1257700 1 --2- 172.21.15.150:0/3281244888 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a9c105f00 0x7f4a9c1082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:48.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.966+0000 7f4aa1257700 1 -- 172.21.15.150:0/3281244888 >> 172.21.15.150:0/3281244888 conn(0x7f4a9c0fcf60 msgr2=0x7f4a9c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:00:48.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.966+0000 7f4aa1257700 1 -- 172.21.15.150:0/3281244888 shutdown_connections 2024-04-25T12:00:48.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.966+0000 7f4aa1257700 1 -- 172.21.15.150:0/3281244888 wait complete. 2024-04-25T12:00:48.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4aa1257700 1 Processor -- start 2024-04-25T12:00:48.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4aa1257700 1 -- start start 2024-04-25T12:00:48.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4aa1257700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a9c1035d0 0x7f4a9c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:00:48.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4aa1257700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c105f00 0x7f4a9c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:00:48.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4aa1257700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a9c108830 0x7f4a9c1a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:00:48.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4aa1257700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c10e650 con 0x7f4a9c105f00 2024-04-25T12:00:48.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4aa1257700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c10e4d0 con 0x7f4a9c108830 2024-04-25T12:00:48.968 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4aa1257700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c10e7d0 con 0x7f4a9c1035d0 2024-04-25T12:00:48.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4a9b59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a9c108830 0x7f4a9c1a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:00:48.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4a9b59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a9c108830 0x7f4a9c1a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:45672/0 (socket says 172.21.15.150:45672) 2024-04-25T12:00:48.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4a9b59e700 1 -- 172.21.15.150:0/567824287 learned_addr learned my addr 172.21.15.150:0/567824287 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T12:00:48.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4a9ad9d700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a9c1035d0 0x7f4a9c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:00:48.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4a93fff700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c105f00 0x7f4a9c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:00:48.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4a9b59e700 1 -- 172.21.15.150:0/567824287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a9c1035d0 msgr2=0x7f4a9c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:00:48.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4a9b59e700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a9c1035d0 0x7f4a9c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:48.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4a9b59e700 1 -- 172.21.15.150:0/567824287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c105f00 msgr2=0x7f4a9c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:00:48.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4a9b59e700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c105f00 0x7f4a9c199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:00:48.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4a9b59e700 1 -- 172.21.15.150:0/567824287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c1a0880 con 0x7f4a9c108830 2024-04-25T12:00:48.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4a9ad9d700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a9c1035d0 0x7f4a9c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:00:48.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4a93fff700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c105f00 0x7f4a9c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:00:48.969 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.967+0000 7f4a9b59e700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a9c108830 0x7f4a9c1a01b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400ada0 tx=0x7f4a9400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:00:48.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.968+0000 7f4a98d99700 1 -- 172.21.15.150:0/567824287 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94018070 con 0x7f4a9c108830 2024-04-25T12:00:48.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.968+0000 7f4a98d99700 1 -- 172.21.15.150:0/567824287 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4a94007e50 con 0x7f4a9c108830 2024-04-25T12:00:48.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.968+0000 7f4aa1257700 1 -- 172.21.15.150:0/567824287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c1a0b70 con 0x7f4a9c108830 2024-04-25T12:00:48.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.968+0000 7f4a98d99700 1 -- 172.21.15.150:0/567824287 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a94004360 con 0x7f4a9c108830 2024-04-25T12:00:48.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.968+0000 7f4aa1257700 1 -- 172.21.15.150:0/567824287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c1a10b0 con 0x7f4a9c108830 2024-04-25T12:00:48.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.969+0000 7f4aa1257700 1 -- 172.21.15.150:0/567824287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a9c069730 con 0x7f4a9c108830 2024-04-25T12:00:48.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.970+0000 7f4a98d99700 1 -- 172.21.15.150:0/567824287 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4a9401b070 con 0x7f4a9c108830 2024-04-25T12:00:48.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.971+0000 7f4a98d99700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4a880a3cc0 0x7f4a880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:00:48.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.971+0000 7f4a98d99700 1 -- 172.21.15.150:0/567824287 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f4a940ce2f0 con 0x7f4a9c108830 2024-04-25T12:00:48.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.971+0000 7f4a9ad9d700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4a880a3cc0 0x7f4a880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:00:48.970 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.971+0000 7f4a9ad9d700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4a880a3cc0 0x7f4a880a6180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c000c00 tx=0x7f4a8c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:00:48.971 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:48.976+0000 7f4a98d99700 1 -- 172.21.15.150:0/567824287 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a9404fff0 con 0x7f4a9c108830 2024-04-25T12:00:49.173 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:00:49.178+0000 7f4aa1257700 1 -- 172.21.15.150:0/567824287 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4a9c062e80 con 0x7f4a880a3cc0 2024-04-25T12:00:49.786 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:00:49.786 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:00:49.787 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:49.794 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:00:49.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:00:49.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:49.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:00:49.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:00:49.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:00:50.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:50 smithi019 ceph-mon[29213]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 130 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:00:50.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:50 smithi019 ceph-mon[29213]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:00:50.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:50 smithi019 ceph-mon[29213]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T12:00:50.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:50 smithi150 ceph-mon[32250]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 130 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:00:50.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:50 smithi150 ceph-mon[32250]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:00:50.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:50 smithi150 ceph-mon[32250]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T12:00:50.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:50 smithi169 ceph-mon[32710]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 130 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:00:50.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:50 smithi169 ceph-mon[32710]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:00:50.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:50 smithi169 ceph-mon[32710]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T12:00:51.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:51 smithi019 ceph-mon[29213]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T12:00:51.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:51 smithi169 ceph-mon[32710]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T12:00:51.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:51 smithi150 ceph-mon[32250]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T12:00:52.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:52 smithi019 ceph-mon[29213]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 130 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:00:52.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:52 smithi019 ceph-mon[29213]: from='client.? 172.21.15.150:0/436270086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14ddc9b5-1675-428c-b759-44c255d90bac"}]: dispatch 2024-04-25T12:00:52.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:52 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14ddc9b5-1675-428c-b759-44c255d90bac"}]: dispatch 2024-04-25T12:00:52.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:52 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14ddc9b5-1675-428c-b759-44c255d90bac"}]': finished 2024-04-25T12:00:52.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:52 smithi019 ceph-mon[29213]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T12:00:52.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:52 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:00:52.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:52 smithi169 ceph-mon[32710]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 130 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:00:52.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:52 smithi169 ceph-mon[32710]: from='client.? 172.21.15.150:0/436270086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14ddc9b5-1675-428c-b759-44c255d90bac"}]: dispatch 2024-04-25T12:00:52.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:52 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14ddc9b5-1675-428c-b759-44c255d90bac"}]: dispatch 2024-04-25T12:00:52.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:52 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14ddc9b5-1675-428c-b759-44c255d90bac"}]': finished 2024-04-25T12:00:52.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:52 smithi169 ceph-mon[32710]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T12:00:52.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:52 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:00:52.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:52 smithi150 ceph-mon[32250]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 130 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:00:52.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:52 smithi150 ceph-mon[32250]: from='client.? 172.21.15.150:0/436270086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14ddc9b5-1675-428c-b759-44c255d90bac"}]: dispatch 2024-04-25T12:00:52.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:52 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14ddc9b5-1675-428c-b759-44c255d90bac"}]: dispatch 2024-04-25T12:00:52.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:52 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14ddc9b5-1675-428c-b759-44c255d90bac"}]': finished 2024-04-25T12:00:52.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:52 smithi150 ceph-mon[32250]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T12:00:52.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:52 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:00:53.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:53 smithi019 ceph-mon[29213]: from='client.? 172.21.15.150:0/2029160881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:00:53.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:53 smithi169 ceph-mon[32710]: from='client.? 172.21.15.150:0/2029160881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:00:53.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:53 smithi150 ceph-mon[32250]: from='client.? 172.21.15.150:0/2029160881' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:00:54.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:54 smithi019 ceph-mon[29213]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:00:54.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:54 smithi169 ceph-mon[32710]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:00:54.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:54 smithi150 ceph-mon[32250]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:00:56.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:56 smithi019 ceph-mon[29213]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T12:00:56.866 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:56 smithi150 ceph-mon[32250]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T12:00:56.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:56 smithi169 ceph-mon[32710]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T12:00:58.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:00:58 smithi019 ceph-mon[29213]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-25T12:00:58.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:00:58 smithi169 ceph-mon[32710]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-25T12:00:58.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:00:58 smithi150 ceph-mon[32250]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-25T12:01:00.514 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:00 smithi150 ceph-mon[32250]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-25T12:01:00.515 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T12:01:00.515 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:00.515 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:00 smithi150 ceph-mon[32250]: Deploying daemon osd.4 on smithi150 2024-04-25T12:01:00.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:00 smithi019 ceph-mon[29213]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-25T12:01:00.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T12:01:00.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:00.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:00 smithi019 ceph-mon[29213]: Deploying daemon osd.4 on smithi150 2024-04-25T12:01:00.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:00 smithi169 ceph-mon[32710]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-25T12:01:00.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T12:01:00.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:00.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:00 smithi169 ceph-mon[32710]: Deploying daemon osd.4 on smithi150 2024-04-25T12:01:02.650 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:02 smithi150 ceph-mon[32250]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T12:01:02.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:02 smithi019 ceph-mon[29213]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T12:01:02.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:02 smithi169 ceph-mon[32710]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T12:01:04.725 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:04 smithi150 ceph-mon[32250]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T12:01:04.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:04 smithi019 ceph-mon[29213]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T12:01:04.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:04 smithi169 ceph-mon[32710]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T12:01:06.650 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:06 smithi150 ceph-mon[32250]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T12:01:06.650 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:06 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:06.650 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:06 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:06.650 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:06 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:06.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:06 smithi169 ceph-mon[32710]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T12:01:06.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:06 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:06.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:06 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:06.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:06 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:06.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:06 smithi019 ceph-mon[29213]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T12:01:06.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:06 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:06.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:06 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:06.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:06 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:08.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:08 smithi169 ceph-mon[32710]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:01:08.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:08.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:08.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:08.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:01:08.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:08.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:08 smithi150 ceph-mon[32250]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:01:08.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:08.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:08.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:08.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:01:08.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:08.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:08 smithi019 ceph-mon[29213]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:01:08.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:08.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:08.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:08.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:01:08.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:09.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:01:09.452+0000 7f4a98d99700 1 -- 172.21.15.150:0/567824287 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4a9c062e80 con 0x7f4a880a3cc0 2024-04-25T12:01:09.450 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 4 on host 'smithi150' 2024-04-25T12:01:09.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:01:09.454+0000 7f4aa1257700 1 -- 172.21.15.150:0/567824287 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4a880a3cc0 msgr2=0x7f4a880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:01:09.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:01:09.454+0000 7f4aa1257700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4a880a3cc0 0x7f4a880a6180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c000c00 tx=0x7f4a8c005fb0 comp rx=0 tx=0).stop 2024-04-25T12:01:09.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:01:09.455+0000 7f4aa1257700 1 -- 172.21.15.150:0/567824287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a9c108830 msgr2=0x7f4a9c1a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:01:09.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:01:09.455+0000 7f4aa1257700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a9c108830 0x7f4a9c1a01b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400ada0 tx=0x7f4a9400c330 comp rx=0 tx=0).stop 2024-04-25T12:01:09.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:01:09.455+0000 7f4aa1257700 1 -- 172.21.15.150:0/567824287 shutdown_connections 2024-04-25T12:01:09.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:01:09.455+0000 7f4aa1257700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4a880a3cc0 0x7f4a880a6180 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:09.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:01:09.455+0000 7f4aa1257700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a9c108830 0x7f4a9c1a01b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:09.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:01:09.455+0000 7f4aa1257700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a9c1035d0 0x7f4a9c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:09.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:01:09.455+0000 7f4aa1257700 1 --2- 172.21.15.150:0/567824287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c105f00 0x7f4a9c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:09.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:01:09.455+0000 7f4aa1257700 1 -- 172.21.15.150:0/567824287 >> 172.21.15.150:0/567824287 conn(0x7f4a9c0fcf60 msgr2=0x7f4a9c103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:01:09.451 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:01:09.455+0000 7f4aa1257700 1 -- 172.21.15.150:0/567824287 shutdown_connections 2024-04-25T12:01:09.452 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:01:09.455+0000 7f4aa1257700 1 -- 172.21.15.150:0/567824287 wait complete. 2024-04-25T12:01:09.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:09 smithi150 ceph-mon[32250]: from='osd.4 [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:01:09.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:09 smithi150 ceph-mon[32250]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:01:09.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:09 smithi019 ceph-mon[29213]: from='osd.4 [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:01:09.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:09 smithi019 ceph-mon[29213]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:01:09.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:09 smithi169 ceph-mon[32710]: from='osd.4 [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:01:09.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:09 smithi169 ceph-mon[32710]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T12:01:10.186 DEBUG:teuthology.orchestra.run.smithi150:osd.4> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T12:01:10.189 INFO:tasks.cephadm:Deploying osd.5 on smithi169 with /dev/vg_nvme/lv_3... 2024-04-25T12:01:10.189 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T12:01:10.435 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:01:10.557 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:10 smithi169 ceph-mon[32710]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:01:10.557 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:10 smithi169 ceph-mon[32710]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T12:01:10.557 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:10 smithi169 ceph-mon[32710]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T12:01:10.557 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:10 smithi169 ceph-mon[32710]: from='osd.4 [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:01:10.557 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:10 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:01:10.558 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:10 smithi169 ceph-mon[32710]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:01:10.558 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:10 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:10.558 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:10 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:10.558 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:10 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:10.660 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:10 smithi150 ceph-mon[32250]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:01:10.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:10 smithi150 ceph-mon[32250]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T12:01:10.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:10 smithi150 ceph-mon[32250]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T12:01:10.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:10 smithi150 ceph-mon[32250]: from='osd.4 [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:01:10.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:10 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:01:10.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:10 smithi150 ceph-mon[32250]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:01:10.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:10 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:10.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:10 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:10.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:10 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:10.661 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:01:10 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4[43826]: 2024-04-25T12:01:10.416+0000 7fb3426e1700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T12:01:10.662 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:01:10 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4[43826]: 2024-04-25T12:01:10.422+0000 7fb33e4d4700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:01:10.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:10 smithi019 ceph-mon[29213]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:01:10.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:10 smithi019 ceph-mon[29213]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T12:01:10.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:10 smithi019 ceph-mon[29213]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T12:01:10.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:10 smithi019 ceph-mon[29213]: from='osd.4 [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:01:10.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:10 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:01:10.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:10 smithi019 ceph-mon[29213]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:01:10.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:10 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:10.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:10 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:10.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:10 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:11.762 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:11 smithi169 ceph-mon[32710]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T12:01:11.762 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:11 smithi169 ceph-mon[32710]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T12:01:11.762 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:11 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:01:11.762 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:11 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:01:11.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:11 smithi150 ceph-mon[32250]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T12:01:11.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:11 smithi150 ceph-mon[32250]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T12:01:11.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:11 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:01:11.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:11 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:01:11.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:11 smithi019 ceph-mon[29213]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T12:01:11.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:11 smithi019 ceph-mon[29213]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T12:01:11.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:11 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:01:11.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:11 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:01:12.305 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T12:01:12.305 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T12:01:12.305 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T12:01:12.305 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T12:01:12.305 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011515 s, 911 MB/s 2024-04-25T12:01:12.305 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T12:01:12.678 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:12 smithi150 ceph-mon[32250]: purged_snaps scrub starts 2024-04-25T12:01:12.678 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:12 smithi150 ceph-mon[32250]: purged_snaps scrub ok 2024-04-25T12:01:12.678 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:12 smithi150 ceph-mon[32250]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:01:12.678 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:12 smithi150 ceph-mon[32250]: osd.4 [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887] boot 2024-04-25T12:01:12.678 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:12 smithi150 ceph-mon[32250]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T12:01:12.678 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:12 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:01:12.768 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:12 smithi169 ceph-mon[32710]: purged_snaps scrub starts 2024-04-25T12:01:12.768 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:12 smithi169 ceph-mon[32710]: purged_snaps scrub ok 2024-04-25T12:01:12.768 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:12 smithi169 ceph-mon[32710]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:01:12.768 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:12 smithi169 ceph-mon[32710]: osd.4 [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887] boot 2024-04-25T12:01:12.768 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:12 smithi169 ceph-mon[32710]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T12:01:12.768 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:12 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:01:12.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:12 smithi019 ceph-mon[29213]: purged_snaps scrub starts 2024-04-25T12:01:12.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:12 smithi019 ceph-mon[29213]: purged_snaps scrub ok 2024-04-25T12:01:12.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:12 smithi019 ceph-mon[29213]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T12:01:12.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:12 smithi019 ceph-mon[29213]: osd.4 [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887] boot 2024-04-25T12:01:12.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:12 smithi019 ceph-mon[29213]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T12:01:12.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:12 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:01:12.892 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_3 2024-04-25T12:01:13.131 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:01:13.526 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:13 smithi169 ceph-mon[32710]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T12:01:13.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:13 smithi019 ceph-mon[29213]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T12:01:13.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:13 smithi150 ceph-mon[32250]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T12:01:14.590 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.592+0000 7fc07a88e700 1 -- 172.21.15.169:0/1817898749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc074104640 msgr2=0x7fc07410aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:01:14.590 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.592+0000 7fc07a88e700 1 --2- 172.21.15.169:0/1817898749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc074104640 0x7fc07410aef0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00b0a0 tx=0x7fc06c00b3b0 comp rx=0 tx=0).stop 2024-04-25T12:01:14.590 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.592+0000 7fc07a88e700 1 -- 172.21.15.169:0/1817898749 shutdown_connections 2024-04-25T12:01:14.590 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.592+0000 7fc07a88e700 1 --2- 172.21.15.169:0/1817898749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc074102aa0 0x7fc074102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:14.590 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.592+0000 7fc07a88e700 1 --2- 172.21.15.169:0/1817898749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc074104640 0x7fc07410aef0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:14.590 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.592+0000 7fc07a88e700 1 --2- 172.21.15.169:0/1817898749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc074103ca0 0x7fc074104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.592+0000 7fc07a88e700 1 -- 172.21.15.169:0/1817898749 >> 172.21.15.169:0/1817898749 conn(0x7fc0740fe040 msgr2=0x7fc074100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:01:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.592+0000 7fc07a88e700 1 -- 172.21.15.169:0/1817898749 shutdown_connections 2024-04-25T12:01:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.592+0000 7fc07a88e700 1 -- 172.21.15.169:0/1817898749 wait complete. 2024-04-25T12:01:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc07a88e700 1 Processor -- start 2024-04-25T12:01:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc07a88e700 1 -- start start 2024-04-25T12:01:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc07a88e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc074102aa0 0x7fc074198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:01:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc07a88e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc074103ca0 0x7fc0741991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:01:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc07a88e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc074104640 0x7fc0741a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:01:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc07a88e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc07410e680 con 0x7fc074104640 2024-04-25T12:01:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc07a88e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc07410e500 con 0x7fc074102aa0 2024-04-25T12:01:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc07a88e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc07410e800 con 0x7fc074103ca0 2024-04-25T12:01:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc06bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc074103ca0 0x7fc0741991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:01:14.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc06bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc074103ca0 0x7fc0741991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:60502/0 (socket says 172.21.15.169:60502) 2024-04-25T12:01:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc06bfff700 1 -- 172.21.15.169:0/3102860892 learned_addr learned my addr 172.21.15.169:0/3102860892 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T12:01:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc078e2b700 1 --2- 172.21.15.169:0/3102860892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc074104640 0x7fc0741a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:01:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc073fff700 1 --2- 172.21.15.169:0/3102860892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc074102aa0 0x7fc074198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:01:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc06bfff700 1 -- 172.21.15.169:0/3102860892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc074102aa0 msgr2=0x7fc074198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:01:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc06bfff700 1 --2- 172.21.15.169:0/3102860892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc074102aa0 0x7fc074198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc06bfff700 1 -- 172.21.15.169:0/3102860892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc074104640 msgr2=0x7fc0741a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:01:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc06bfff700 1 --2- 172.21.15.169:0/3102860892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc074104640 0x7fc0741a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.593+0000 7fc06bfff700 1 -- 172.21.15.169:0/3102860892 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0741a08b0 con 0x7fc074103ca0 2024-04-25T12:01:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.594+0000 7fc06bfff700 1 --2- 172.21.15.169:0/3102860892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc074103ca0 0x7fc0741991a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00d980 tx=0x7fc05c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:01:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.594+0000 7fc071ffb700 1 -- 172.21.15.169:0/3102860892 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c017070 con 0x7fc074103ca0 2024-04-25T12:01:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.594+0000 7fc071ffb700 1 -- 172.21.15.169:0/3102860892 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc05c005020 con 0x7fc074103ca0 2024-04-25T12:01:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.594+0000 7fc07a88e700 1 -- 172.21.15.169:0/3102860892 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0741a0ba0 con 0x7fc074103ca0 2024-04-25T12:01:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.594+0000 7fc071ffb700 1 -- 172.21.15.169:0/3102860892 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c009bf0 con 0x7fc074103ca0 2024-04-25T12:01:14.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.594+0000 7fc07a88e700 1 -- 172.21.15.169:0/3102860892 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0741a10e0 con 0x7fc074103ca0 2024-04-25T12:01:14.593 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.595+0000 7fc07a88e700 1 -- 172.21.15.169:0/3102860892 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc074069730 con 0x7fc074103ca0 2024-04-25T12:01:14.593 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.596+0000 7fc071ffb700 1 -- 172.21.15.169:0/3102860892 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc05c0051d0 con 0x7fc074103ca0 2024-04-25T12:01:14.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.596+0000 7fc071ffb700 1 --2- 172.21.15.169:0/3102860892 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc0600a3cc0 0x7fc0600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:01:14.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.596+0000 7fc071ffb700 1 -- 172.21.15.169:0/3102860892 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fc05c0cf1b0 con 0x7fc074103ca0 2024-04-25T12:01:14.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.596+0000 7fc073fff700 1 --2- 172.21.15.169:0/3102860892 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc0600a3cc0 0x7fc0600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:01:14.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.597+0000 7fc073fff700 1 --2- 172.21.15.169:0/3102860892 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc0600a3cc0 0x7fc0600a6180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc07419a140 tx=0x7fc06400b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:01:14.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.601+0000 7fc071ffb700 1 -- 172.21.15.169:0/3102860892 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc05c020020 con 0x7fc074103ca0 2024-04-25T12:01:14.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:14 smithi169 ceph-mon[32710]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:14.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:14 smithi169 ceph-mon[32710]: Detected new or changed devices on smithi150 2024-04-25T12:01:14.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:14 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:14.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:14 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:14.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:14 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:01:14.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:14 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:01:14.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:14 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi150 to 8544M 2024-04-25T12:01:14.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:14 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:14.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:14 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:14.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:14 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:01:14.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:14 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:14.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:14 smithi019 ceph-mon[29213]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:14.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:14 smithi019 ceph-mon[29213]: Detected new or changed devices on smithi150 2024-04-25T12:01:14.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:14 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:14.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:14 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:14.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:14 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:01:14.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:14 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:01:14.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:14 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi150 to 8544M 2024-04-25T12:01:14.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:14 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:14.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:14 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:14.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:14 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:01:14.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:14 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:14.804 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:14.809+0000 7fc07a88e700 1 -- 172.21.15.169:0/3102860892 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc074062e80 con 0x7fc0600a3cc0 2024-04-25T12:01:14.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:14 smithi150 ceph-mon[32250]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:14.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:14 smithi150 ceph-mon[32250]: Detected new or changed devices on smithi150 2024-04-25T12:01:14.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:14 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:14.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:14 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:14.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:14 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:01:14.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:14 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:01:14.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:14 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi150 to 8544M 2024-04-25T12:01:14.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:14 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:14.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:14 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:14.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:14 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:01:14.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:14 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:15.729 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:01:15.729 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:01:15.729 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:15.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:01:15.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:01:15.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:15.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:01:15.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:01:15.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:16.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:16 smithi169 ceph-mon[32710]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:01:16.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:16 smithi169 ceph-mon[32710]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:16.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:16 smithi019 ceph-mon[29213]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:01:16.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:16 smithi019 ceph-mon[29213]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:16.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:16 smithi150 ceph-mon[32250]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:01:16.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:16 smithi150 ceph-mon[32250]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:17.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:17 smithi019 ceph-mon[29213]: from='client.? 172.21.15.169:0/569745194' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b7ca08-7d93-405d-9f28-9a98b7b3f669"}]: dispatch 2024-04-25T12:01:17.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:17 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b7ca08-7d93-405d-9f28-9a98b7b3f669"}]: dispatch 2024-04-25T12:01:17.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:17 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99b7ca08-7d93-405d-9f28-9a98b7b3f669"}]': finished 2024-04-25T12:01:17.872 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:17 smithi169 ceph-mon[32710]: from='client.? 172.21.15.169:0/569745194' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b7ca08-7d93-405d-9f28-9a98b7b3f669"}]: dispatch 2024-04-25T12:01:17.873 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:17 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b7ca08-7d93-405d-9f28-9a98b7b3f669"}]: dispatch 2024-04-25T12:01:17.873 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:17 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99b7ca08-7d93-405d-9f28-9a98b7b3f669"}]': finished 2024-04-25T12:01:17.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:17 smithi150 ceph-mon[32250]: from='client.? 172.21.15.169:0/569745194' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b7ca08-7d93-405d-9f28-9a98b7b3f669"}]: dispatch 2024-04-25T12:01:17.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:17 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b7ca08-7d93-405d-9f28-9a98b7b3f669"}]: dispatch 2024-04-25T12:01:17.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:17 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99b7ca08-7d93-405d-9f28-9a98b7b3f669"}]': finished 2024-04-25T12:01:18.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:18 smithi019 ceph-mon[29213]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:18.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:18 smithi019 ceph-mon[29213]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T12:01:18.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:18.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:18 smithi019 ceph-mon[29213]: from='client.? 172.21.15.169:0/4184896143' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:01:18.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:18 smithi169 ceph-mon[32710]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:18.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:18 smithi169 ceph-mon[32710]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T12:01:18.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:18 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:18.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:18 smithi169 ceph-mon[32710]: from='client.? 172.21.15.169:0/4184896143' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:01:18.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:18 smithi150 ceph-mon[32250]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:18.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:18 smithi150 ceph-mon[32250]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T12:01:18.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:18 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:18.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:18 smithi150 ceph-mon[32250]: from='client.? 172.21.15.169:0/4184896143' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:01:20.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:20 smithi019 ceph-mon[29213]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:20.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:20 smithi169 ceph-mon[32710]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:20.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:20 smithi150 ceph-mon[32250]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:22.720 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:22 smithi169 ceph-mon[32710]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:22.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:22 smithi019 ceph-mon[29213]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:22.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:22 smithi150 ceph-mon[32250]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:24.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:24 smithi019 ceph-mon[29213]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:24.840 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:24 smithi169 ceph-mon[32710]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:24.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:24 smithi150 ceph-mon[32250]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:25.785 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:25 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T12:01:25.785 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:25 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:25.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:25 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T12:01:25.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:25 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:26.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:25 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T12:01:26.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:25 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:26.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:26 smithi169 ceph-mon[32710]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:26.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:26 smithi169 ceph-mon[32710]: Deploying daemon osd.5 on smithi169 2024-04-25T12:01:26.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:26 smithi150 ceph-mon[32250]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:26.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:26 smithi150 ceph-mon[32250]: Deploying daemon osd.5 on smithi169 2024-04-25T12:01:27.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:26 smithi019 ceph-mon[29213]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:27.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:26 smithi019 ceph-mon[29213]: Deploying daemon osd.5 on smithi169 2024-04-25T12:01:28.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:28 smithi169 ceph-mon[32710]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:28.910 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:28 smithi150 ceph-mon[32250]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:29.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:28 smithi019 ceph-mon[29213]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:30.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:30 smithi169 ceph-mon[32710]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:30.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:30 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:30.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:30 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:30.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:30 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:30.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:30 smithi150 ceph-mon[32250]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:30.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:30 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:30.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:30 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:30.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:30 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:31.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:30 smithi019 ceph-mon[29213]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:31.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:30 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:31.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:30 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:31.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:30 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:32.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:32 smithi169 ceph-mon[32710]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:32.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:32 smithi150 ceph-mon[32250]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:33.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:32 smithi019 ceph-mon[29213]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:33.721 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:33.725+0000 7fc071ffb700 1 -- 172.21.15.169:0/3102860892 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc074062e80 con 0x7fc0600a3cc0 2024-04-25T12:01:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 5 on host 'smithi169' 2024-04-25T12:01:33.723 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:33.727+0000 7fc07a88e700 1 -- 172.21.15.169:0/3102860892 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc0600a3cc0 msgr2=0x7fc0600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:01:33.724 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:33.727+0000 7fc07a88e700 1 --2- 172.21.15.169:0/3102860892 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc0600a3cc0 0x7fc0600a6180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc07419a140 tx=0x7fc06400b5d0 comp rx=0 tx=0).stop 2024-04-25T12:01:33.724 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:33.727+0000 7fc07a88e700 1 -- 172.21.15.169:0/3102860892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc074103ca0 msgr2=0x7fc0741991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:01:33.724 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:33.727+0000 7fc07a88e700 1 --2- 172.21.15.169:0/3102860892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc074103ca0 0x7fc0741991a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00d980 tx=0x7fc05c00dc90 comp rx=0 tx=0).stop 2024-04-25T12:01:33.724 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:33.727+0000 7fc07a88e700 1 -- 172.21.15.169:0/3102860892 shutdown_connections 2024-04-25T12:01:33.724 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:33.727+0000 7fc07a88e700 1 --2- 172.21.15.169:0/3102860892 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc0600a3cc0 0x7fc0600a6180 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:33.724 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:33.727+0000 7fc07a88e700 1 --2- 172.21.15.169:0/3102860892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc074104640 0x7fc0741a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:33.724 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:33.727+0000 7fc07a88e700 1 --2- 172.21.15.169:0/3102860892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc074102aa0 0x7fc074198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:33.724 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:33.727+0000 7fc07a88e700 1 --2- 172.21.15.169:0/3102860892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc074103ca0 0x7fc0741991a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:33.724 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:33.727+0000 7fc07a88e700 1 -- 172.21.15.169:0/3102860892 >> 172.21.15.169:0/3102860892 conn(0x7fc0740fe040 msgr2=0x7fc074107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:01:33.724 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:33.728+0000 7fc07a88e700 1 -- 172.21.15.169:0/3102860892 shutdown_connections 2024-04-25T12:01:33.724 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:01:33.728+0000 7fc07a88e700 1 -- 172.21.15.169:0/3102860892 wait complete. 2024-04-25T12:01:34.402 DEBUG:teuthology.orchestra.run.smithi169:osd.5> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T12:01:34.404 INFO:tasks.cephadm:Deploying osd.6 on smithi019 with /dev/vg_nvme/lv_2... 2024-04-25T12:01:34.404 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T12:01:34.662 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:01:34.992 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:34 smithi169 ceph-mon[32710]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:34.992 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:34 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:34.992 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:34 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:34.992 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:34 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:34.992 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:34 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:34.992 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:34 smithi169 ceph-mon[32710]: from='osd.5 [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:01:34.993 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:34 smithi169 ceph-mon[32710]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:01:35.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:34 smithi019 ceph-mon[29213]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:35.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:34 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:35.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:34 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:35.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:34 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:35.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:34 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:35.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:34 smithi019 ceph-mon[29213]: from='osd.5 [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:01:35.004 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:34 smithi019 ceph-mon[29213]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:01:35.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:34 smithi150 ceph-mon[32250]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:35.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:34 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:35.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:34 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:35.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:34 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:35.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:34 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:35.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:34 smithi150 ceph-mon[32250]: from='osd.5 [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:01:35.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:34 smithi150 ceph-mon[32250]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T12:01:36.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:36 smithi019 ceph-mon[29213]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:36.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:36 smithi019 ceph-mon[29213]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T12:01:36.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:36 smithi019 ceph-mon[29213]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T12:01:36.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:36 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:36.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:36 smithi019 ceph-mon[29213]: from='osd.5 [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:01:36.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:36 smithi019 ceph-mon[29213]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:01:36.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:36 smithi169 ceph-mon[32710]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:36.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:36 smithi169 ceph-mon[32710]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T12:01:36.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:36 smithi169 ceph-mon[32710]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T12:01:36.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:36 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:36.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:36 smithi169 ceph-mon[32710]: from='osd.5 [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:01:36.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:36 smithi169 ceph-mon[32710]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:01:36.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:36 smithi150 ceph-mon[32250]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:36.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:36 smithi150 ceph-mon[32250]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T12:01:36.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:36 smithi150 ceph-mon[32250]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T12:01:36.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:36 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:36.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:36 smithi150 ceph-mon[32250]: from='osd.5 [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:01:36.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:36 smithi150 ceph-mon[32250]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:01:36.735 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:01:36 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:01:36.533+0000 7f72d8c4d700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T12:01:36.735 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:01:36 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:01:36.539+0000 7f72d0237700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:01:36.903 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T12:01:36.904 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T12:01:36.904 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-25T12:01:36.904 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-25T12:01:36.904 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114351 s, 917 MB/s 2024-04-25T12:01:36.904 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-25T12:01:37.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:37 smithi169 ceph-mon[32710]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T12:01:37.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:37 smithi169 ceph-mon[32710]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T12:01:37.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:37 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:37.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:37 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:37.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:37 smithi169 ceph-mon[32710]: from='osd.5 ' entity='osd.5' 2024-04-25T12:01:37.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:37 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:37.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:37 smithi150 ceph-mon[32250]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T12:01:37.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:37 smithi150 ceph-mon[32250]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T12:01:37.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:37 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:37.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:37 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:37.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:37 smithi150 ceph-mon[32250]: from='osd.5 ' entity='osd.5' 2024-04-25T12:01:37.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:37 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:37.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:37 smithi019 ceph-mon[29213]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T12:01:37.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:37 smithi019 ceph-mon[29213]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T12:01:37.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:37 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:37.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:37 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:37.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:37 smithi019 ceph-mon[29213]: from='osd.5 ' entity='osd.5' 2024-04-25T12:01:37.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:37 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:37.601 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_2 2024-04-25T12:01:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:01:38.234 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: purged_snaps scrub starts 2024-04-25T12:01:38.234 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: purged_snaps scrub ok 2024-04-25T12:01:38.234 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:38.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: osd.5 [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081] boot 2024-04-25T12:01:38.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T12:01:38.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:38.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: Detected new or changed devices on smithi169 2024-04-25T12:01:38.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:38.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:38.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:01:38.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:01:38.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-25T12:01:38.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:38.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:38.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:01:38.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:38.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:38.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:38.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:01:38.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:38.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: purged_snaps scrub starts 2024-04-25T12:01:38.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: purged_snaps scrub ok 2024-04-25T12:01:38.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:38.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: osd.5 [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081] boot 2024-04-25T12:01:38.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T12:01:38.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:38.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: Detected new or changed devices on smithi169 2024-04-25T12:01:38.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:38.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:38.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:01:38.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:01:38.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-25T12:01:38.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:38.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:38.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:01:38.407 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:38.407 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:38.407 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:38.407 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:01:38.407 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:38.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: purged_snaps scrub starts 2024-04-25T12:01:38.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: purged_snaps scrub ok 2024-04-25T12:01:38.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T12:01:38.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: osd.5 [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081] boot 2024-04-25T12:01:38.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T12:01:38.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:01:38.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: Detected new or changed devices on smithi169 2024-04-25T12:01:38.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:38.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:38.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:01:38.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:01:38.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi169 to 8544M 2024-04-25T12:01:38.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:38.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:38.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:01:38.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:38.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:38.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:38.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:01:38.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:39.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:39 smithi019 ceph-mon[29213]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T12:01:39.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:39 smithi169 ceph-mon[32710]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T12:01:39.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:39 smithi150 ceph-mon[32250]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T12:01:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.560+0000 7f06b57b4700 1 -- 172.21.15.19:0/1372137677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b0108780 msgr2=0x7f06b010abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:01:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.560+0000 7f06b57b4700 1 --2- 172.21.15.19:0/1372137677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b0108780 0x7f06b010abf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f06a800b0a0 tx=0x7f06a800b3b0 comp rx=0 tx=0).stop 2024-04-25T12:01:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.560+0000 7f06b57b4700 1 -- 172.21.15.19:0/1372137677 shutdown_connections 2024-04-25T12:01:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.560+0000 7f06b57b4700 1 --2- 172.21.15.19:0/1372137677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b0108780 0x7f06b010abf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.560+0000 7f06b57b4700 1 --2- 172.21.15.19:0/1372137677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06b0103520 0x7f06b0105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.560+0000 7f06b57b4700 1 --2- 172.21.15.19:0/1372137677 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06b0105e50 0x7f06b0108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.560+0000 7f06b57b4700 1 -- 172.21.15.19:0/1372137677 >> 172.21.15.19:0/1372137677 conn(0x7f06b00fcf30 msgr2=0x7f06b00ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:01:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.560+0000 7f06b57b4700 1 -- 172.21.15.19:0/1372137677 shutdown_connections 2024-04-25T12:01:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.560+0000 7f06b57b4700 1 -- 172.21.15.19:0/1372137677 wait complete. 2024-04-25T12:01:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.561+0000 7f06b57b4700 1 Processor -- start 2024-04-25T12:01:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.561+0000 7f06b57b4700 1 -- start start 2024-04-25T12:01:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.561+0000 7f06b57b4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b0103520 0x7f06b0102e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:01:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.561+0000 7f06b57b4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06b0105e50 0x7f06b01014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:01:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.561+0000 7f06b57b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06b0108780 0x7f06b0101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:01:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.561+0000 7f06b57b4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06b010e640 con 0x7f06b0103520 2024-04-25T12:01:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.561+0000 7f06b57b4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f06b010e4c0 con 0x7f06b0108780 2024-04-25T12:01:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.561+0000 7f06b57b4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f06b010e7c0 con 0x7f06b0105e50 2024-04-25T12:01:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.561+0000 7f06aeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b0103520 0x7f06b0102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:01:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.561+0000 7f06aeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b0103520 0x7f06b0102e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39676/0 (socket says 172.21.15.19:39676) 2024-04-25T12:01:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.561+0000 7f06aeffd700 1 -- 172.21.15.19:0/2558618540 learned_addr learned my addr 172.21.15.19:0/2558618540 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:01:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.561+0000 7f06ae7fc700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06b0105e50 0x7f06b01014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:01:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.561+0000 7f06af7fe700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06b0108780 0x7f06b0101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:01:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.562+0000 7f06aeffd700 1 -- 172.21.15.19:0/2558618540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06b0105e50 msgr2=0x7f06b01014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:01:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.562+0000 7f06aeffd700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06b0105e50 0x7f06b01014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.562+0000 7f06aeffd700 1 -- 172.21.15.19:0/2558618540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06b0108780 msgr2=0x7f06b0101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:01:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.562+0000 7f06aeffd700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06b0108780 0x7f06b0101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:01:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.562+0000 7f06aeffd700 1 -- 172.21.15.19:0/2558618540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b01022c0 con 0x7f06b0103520 2024-04-25T12:01:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.562+0000 7f06af7fe700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06b0108780 0x7f06b0101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:01:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.562+0000 7f06aeffd700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b0103520 0x7f06b0102e80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f06a000d980 tx=0x7f06a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:01:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.562+0000 7f0697fff700 1 -- 172.21.15.19:0/2558618540 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0016070 con 0x7f06b0103520 2024-04-25T12:01:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.562+0000 7f06b57b4700 1 -- 172.21.15.19:0/2558618540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b01a2dc0 con 0x7f06b0103520 2024-04-25T12:01:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.562+0000 7f0697fff700 1 -- 172.21.15.19:0/2558618540 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f06a0004d00 con 0x7f06b0103520 2024-04-25T12:01:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.562+0000 7f0697fff700 1 -- 172.21.15.19:0/2558618540 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0005690 con 0x7f06b0103520 2024-04-25T12:01:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.562+0000 7f06b57b4700 1 -- 172.21.15.19:0/2558618540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b01a3210 con 0x7f06b0103520 2024-04-25T12:01:39.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.563+0000 7f06b57b4700 1 -- 172.21.15.19:0/2558618540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b0069730 con 0x7f06b0103520 2024-04-25T12:01:39.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.564+0000 7f0697fff700 1 -- 172.21.15.19:0/2558618540 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f06a0004750 con 0x7f06b0103520 2024-04-25T12:01:39.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.565+0000 7f0697fff700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f06980a3c40 0x7f06980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:01:39.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.565+0000 7f06ae7fc700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f06980a3c40 0x7f06980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:01:39.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.565+0000 7f0697fff700 1 -- 172.21.15.19:0/2558618540 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f06a00cf8a0 con 0x7f06b0103520 2024-04-25T12:01:39.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.566+0000 7f06ae7fc700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f06980a3c40 0x7f06980a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009be0 tx=0x7f06a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:01:39.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.569+0000 7f0697fff700 1 -- 172.21.15.19:0/2558618540 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06a001f020 con 0x7f06b0103520 2024-04-25T12:01:39.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:01:39.789+0000 7f06b57b4700 1 -- 172.21.15.19:0/2558618540 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f06b0062e80 con 0x7f06980a3c40 2024-04-25T12:01:40.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:40 smithi169 ceph-mon[32710]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:01:40.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:40 smithi169 ceph-mon[32710]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T12:01:40.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:01:40.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:01:40.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:40.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:40 smithi150 ceph-mon[32250]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:01:40.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:40 smithi150 ceph-mon[32250]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T12:01:40.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:40 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:01:40.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:40 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:01:40.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:40 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:40 smithi019 ceph-mon[29213]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:01:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:40 smithi019 ceph-mon[29213]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T12:01:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:40 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:01:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:40 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:01:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:40 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:41.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:41 smithi019 ceph-mon[29213]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:01:41.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:41 smithi019 ceph-mon[29213]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T12:01:41.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:41 smithi169 ceph-mon[32710]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:01:41.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:41 smithi169 ceph-mon[32710]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T12:01:41.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:41 smithi150 ceph-mon[32250]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:01:41.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:41 smithi150 ceph-mon[32250]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T12:01:42.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:42 smithi169 ceph-mon[32710]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:01:42.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:42 smithi169 ceph-mon[32710]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T12:01:42.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:42 smithi150 ceph-mon[32250]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:01:42.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:42 smithi150 ceph-mon[32250]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T12:01:42.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:42 smithi019 ceph-mon[29213]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:01:42.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:42 smithi019 ceph-mon[29213]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T12:01:43.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:43 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/110009884' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac"}]: dispatch 2024-04-25T12:01:43.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:43 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/110009884' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac"}]': finished 2024-04-25T12:01:43.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:43 smithi019 ceph-mon[29213]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T12:01:43.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:43 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:01:43.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:43 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/110009884' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac"}]: dispatch 2024-04-25T12:01:43.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:43 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/110009884' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac"}]': finished 2024-04-25T12:01:43.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:43 smithi169 ceph-mon[32710]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T12:01:43.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:43 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:01:43.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:43 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/110009884' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac"}]: dispatch 2024-04-25T12:01:43.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:43 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/110009884' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac"}]': finished 2024-04-25T12:01:43.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:43 smithi150 ceph-mon[32250]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T12:01:43.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:43 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:01:44.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:44 smithi169 ceph-mon[32710]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:01:44.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:44 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1514905829' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:01:44.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:44 smithi150 ceph-mon[32250]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:01:44.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:44 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1514905829' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:01:44.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:44 smithi019 ceph-mon[29213]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:01:44.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:44 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1514905829' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:01:46.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:46 smithi169 ceph-mon[32710]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T12:01:46.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:46 smithi150 ceph-mon[32250]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T12:01:46.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:46 smithi019 ceph-mon[29213]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-25T12:01:48.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:48 smithi019 ceph-mon[29213]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T12:01:48.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:48 smithi169 ceph-mon[32710]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T12:01:48.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:48 smithi150 ceph-mon[32250]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T12:01:50.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:50 smithi169 ceph-mon[32710]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T12:01:50.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:50 smithi150 ceph-mon[32250]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T12:01:50.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:50 smithi019 ceph-mon[29213]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T12:01:51.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:51 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T12:01:51.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:51 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:51.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:51 smithi169 ceph-mon[32710]: Deploying daemon osd.6 on smithi019 2024-04-25T12:01:51.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:51 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T12:01:51.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:51 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:51.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:51 smithi150 ceph-mon[32250]: Deploying daemon osd.6 on smithi019 2024-04-25T12:01:51.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:51 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T12:01:51.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:51 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:01:51.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:51 smithi019 ceph-mon[29213]: Deploying daemon osd.6 on smithi019 2024-04-25T12:01:52.403 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:52 smithi169 ceph-mon[32710]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-25T12:01:52.409 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:52 smithi019 ceph-mon[29213]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-25T12:01:52.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:52 smithi150 ceph-mon[32250]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-25T12:01:54.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:54 smithi019 ceph-mon[29213]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T12:01:54.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:54 smithi150 ceph-mon[32250]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T12:01:54.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:54 smithi169 ceph-mon[32710]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T12:01:56.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:56 smithi150 ceph-mon[32250]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T12:01:56.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:56 smithi019 ceph-mon[29213]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T12:01:56.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:56 smithi169 ceph-mon[32710]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T12:01:57.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:57.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:57.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:57.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:57.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:57.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:57.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:57.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:01:57.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:01:58.470 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:01:58 smithi019 ceph-mon[29213]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:01:58.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:01:58 smithi169 ceph-mon[32710]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:01:58.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:01:58 smithi150 ceph-mon[32250]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:02:00.345 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 6 on host 'smithi019' 2024-04-25T12:02:00.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:00.349+0000 7f0697fff700 1 -- 172.21.15.19:0/2558618540 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f06b0062e80 con 0x7f06980a3c40 2024-04-25T12:02:00.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:00.351+0000 7f06b57b4700 1 -- 172.21.15.19:0/2558618540 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f06980a3c40 msgr2=0x7f06980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:00.351+0000 7f06b57b4700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f06980a3c40 0x7f06980a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009be0 tx=0x7f06a4009260 comp rx=0 tx=0).stop 2024-04-25T12:02:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:00.352+0000 7f06b57b4700 1 -- 172.21.15.19:0/2558618540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b0103520 msgr2=0x7f06b0102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:00.352+0000 7f06b57b4700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b0103520 0x7f06b0102e80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f06a000d980 tx=0x7f06a000dc90 comp rx=0 tx=0).stop 2024-04-25T12:02:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:00.352+0000 7f06b57b4700 1 -- 172.21.15.19:0/2558618540 shutdown_connections 2024-04-25T12:02:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:00.352+0000 7f06b57b4700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f06980a3c40 0x7f06980a6100 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:00.352+0000 7f06b57b4700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06b0103520 0x7f06b0102e80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:00.352+0000 7f06b57b4700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06b0108780 0x7f06b0101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:00.352+0000 7f06b57b4700 1 --2- 172.21.15.19:0/2558618540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06b0105e50 0x7f06b01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:00.352+0000 7f06b57b4700 1 -- 172.21.15.19:0/2558618540 >> 172.21.15.19:0/2558618540 conn(0x7f06b00fcf30 msgr2=0x7f06b0103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:02:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:00.352+0000 7f06b57b4700 1 -- 172.21.15.19:0/2558618540 shutdown_connections 2024-04-25T12:02:00.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:00.352+0000 7f06b57b4700 1 -- 172.21.15.19:0/2558618540 wait complete. 2024-04-25T12:02:00.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:00 smithi019 ceph-mon[29213]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:02:00.545 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:00.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:00.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:00.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:00.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:00.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:00 smithi019 ceph-mon[29213]: from='osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T12:02:00.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:00 smithi169 ceph-mon[32710]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:02:00.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:00.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:00.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:00.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:00.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:00.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:00 smithi169 ceph-mon[32710]: from='osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T12:02:00.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:00 smithi150 ceph-mon[32250]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:02:00.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:00.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:00.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:00.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:00.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:00.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:00 smithi150 ceph-mon[32250]: from='osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T12:02:01.088 DEBUG:teuthology.orchestra.run.smithi019:osd.6> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T12:02:01.090 INFO:tasks.cephadm:Deploying osd.7 on smithi150 with /dev/vg_nvme/lv_2... 2024-04-25T12:02:01.091 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T12:02:01.337 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:02:01.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:01.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:01.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:01 smithi150 ceph-mon[32250]: from='osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T12:02:01.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:01 smithi150 ceph-mon[32250]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T12:02:01.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:01 smithi150 ceph-mon[32250]: from='osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T12:02:01.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:02:01.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:01.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:01.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:01.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:01 smithi019 ceph-mon[29213]: from='osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T12:02:01.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:01 smithi019 ceph-mon[29213]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T12:02:01.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:01 smithi019 ceph-mon[29213]: from='osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T12:02:01.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:02:01.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:01.600 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:02:01 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:02:01.362+0000 7f1c6c2a6700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T12:02:01.600 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:02:01 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:02:01.372+0000 7f1c68099700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:02:01.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:01.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:01.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:01 smithi169 ceph-mon[32710]: from='osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T12:02:01.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:01 smithi169 ceph-mon[32710]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T12:02:01.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:01 smithi169 ceph-mon[32710]: from='osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T12:02:01.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:02:01.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:02.627 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:02 smithi019 ceph-mon[29213]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:02:02.627 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:02 smithi019 ceph-mon[29213]: from='osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T12:02:02.627 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:02 smithi019 ceph-mon[29213]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T12:02:02.627 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:02 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:02:02.627 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:02 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:02:02.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:02 smithi169 ceph-mon[32710]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:02:02.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:02 smithi169 ceph-mon[32710]: from='osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T12:02:02.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:02 smithi169 ceph-mon[32710]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T12:02:02.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:02 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:02:02.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:02 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:02:02.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:02 smithi150 ceph-mon[32250]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T12:02:02.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:02 smithi150 ceph-mon[32250]: from='osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T12:02:02.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:02 smithi150 ceph-mon[32250]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T12:02:02.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:02 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:02:02.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:02 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:02:03.599 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T12:02:03.599 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T12:02:03.599 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-25T12:02:03.600 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-25T12:02:03.600 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011379 s, 922 MB/s 2024-04-25T12:02:03.600 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-25T12:02:03.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:03 smithi150 ceph-mon[32250]: purged_snaps scrub starts 2024-04-25T12:02:03.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:03 smithi150 ceph-mon[32250]: purged_snaps scrub ok 2024-04-25T12:02:03.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:03 smithi150 ceph-mon[32250]: osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084] boot 2024-04-25T12:02:03.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:03 smithi150 ceph-mon[32250]: osdmap e46: 7 total, 7 up, 7 in 2024-04-25T12:02:03.627 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:03 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:02:03.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:03 smithi169 ceph-mon[32710]: purged_snaps scrub starts 2024-04-25T12:02:03.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:03 smithi169 ceph-mon[32710]: purged_snaps scrub ok 2024-04-25T12:02:03.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:03 smithi169 ceph-mon[32710]: osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084] boot 2024-04-25T12:02:03.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:03 smithi169 ceph-mon[32710]: osdmap e46: 7 total, 7 up, 7 in 2024-04-25T12:02:03.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:03 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:02:03.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:03 smithi019 ceph-mon[29213]: purged_snaps scrub starts 2024-04-25T12:02:03.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:03 smithi019 ceph-mon[29213]: purged_snaps scrub ok 2024-04-25T12:02:03.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:03 smithi019 ceph-mon[29213]: osd.6 [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084] boot 2024-04-25T12:02:03.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:03 smithi019 ceph-mon[29213]: osdmap e46: 7 total, 7 up, 7 in 2024-04-25T12:02:03.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:03 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:02:04.249 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2024-04-25T12:02:04.496 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:02:04.614 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:04 smithi150 ceph-mon[32250]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:04.614 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:04 smithi150 ceph-mon[32250]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T12:02:04.653 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:04 smithi169 ceph-mon[32710]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:04.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:04 smithi169 ceph-mon[32710]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T12:02:04.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:04 smithi019 ceph-mon[29213]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:04.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:04 smithi019 ceph-mon[29213]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T12:02:05.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:05 smithi150 ceph-mon[32250]: Detected new or changed devices on smithi019 2024-04-25T12:02:05.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:05 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:05.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:05 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:05.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:05 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:05.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:05 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:05.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:05 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:05.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:05 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi019 to 5697M 2024-04-25T12:02:05.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:05 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:05.630 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:05 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:05.631 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:05 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:05.631 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:05 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:05.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:05 smithi019 ceph-mon[29213]: Detected new or changed devices on smithi019 2024-04-25T12:02:05.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:05 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:05.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:05 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:05.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:05 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:05.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:05 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:05.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:05 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:05.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:05 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi019 to 5697M 2024-04-25T12:02:05.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:05 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:05.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:05 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:05.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:05 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:05.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:05 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:05.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:05 smithi169 ceph-mon[32710]: Detected new or changed devices on smithi019 2024-04-25T12:02:05.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:05 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:05.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:05 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:05.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:05 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:05.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:05 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:05.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:05 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:05.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:05 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi019 to 5697M 2024-04-25T12:02:05.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:05 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:05.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:05 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:05.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:05 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:05.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:05 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:06.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.165+0000 7f8901b68700 1 -- 172.21.15.150:0/183485212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88fc103cb0 msgr2=0x7f88fc104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:06.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.165+0000 7f8901b68700 1 --2- 172.21.15.150:0/183485212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88fc103cb0 0x7f88fc104110 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0099c0 tx=0x7f88ec009cd0 comp rx=0 tx=0).stop 2024-04-25T12:02:06.163 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.165+0000 7f8901b68700 1 -- 172.21.15.150:0/183485212 shutdown_connections 2024-04-25T12:02:06.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.165+0000 7f8901b68700 1 --2- 172.21.15.150:0/183485212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc104650 0x7f88fc10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:06.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.165+0000 7f8901b68700 1 --2- 172.21.15.150:0/183485212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88fc102ab0 0x7f88fc102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:06.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.165+0000 7f8901b68700 1 --2- 172.21.15.150:0/183485212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88fc103cb0 0x7f88fc104110 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:06.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.165+0000 7f8901b68700 1 -- 172.21.15.150:0/183485212 >> 172.21.15.150:0/183485212 conn(0x7f88fc0fe050 msgr2=0x7f88fc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:02:06.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.165+0000 7f8901b68700 1 -- 172.21.15.150:0/183485212 shutdown_connections 2024-04-25T12:02:06.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.165+0000 7f8901b68700 1 -- 172.21.15.150:0/183485212 wait complete. 2024-04-25T12:02:06.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f8901b68700 1 Processor -- start 2024-04-25T12:02:06.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f8901b68700 1 -- start start 2024-04-25T12:02:06.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f8901b68700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc102ab0 0x7f88fc0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:02:06.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f8901b68700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88fc103cb0 0x7f88fc074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:02:06.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f8901b68700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88fc104650 0x7f88fc075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:02:06.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f8901b68700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc10e700 con 0x7f88fc102ab0 2024-04-25T12:02:06.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f8901b68700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc10e580 con 0x7f88fc104650 2024-04-25T12:02:06.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f8901b68700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc10e880 con 0x7f88fc103cb0 2024-04-25T12:02:06.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f88fbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88fc104650 0x7f88fc075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:02:06.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f88fbfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88fc104650 0x7f88fc075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:55342/0 (socket says 172.21.15.150:55342) 2024-04-25T12:02:06.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f88fbfff700 1 -- 172.21.15.150:0/3950059982 learned_addr learned my addr 172.21.15.150:0/3950059982 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T12:02:06.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f88fb7fe700 1 --2- 172.21.15.150:0/3950059982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc102ab0 0x7f88fc0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:02:06.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f88faffd700 1 --2- 172.21.15.150:0/3950059982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88fc103cb0 0x7f88fc074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:02:06.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f88fbfff700 1 -- 172.21.15.150:0/3950059982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88fc103cb0 msgr2=0x7f88fc074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:06.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f88fbfff700 1 --2- 172.21.15.150:0/3950059982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88fc103cb0 0x7f88fc074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:06.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f88fbfff700 1 -- 172.21.15.150:0/3950059982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc102ab0 msgr2=0x7f88fc0767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:06.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f88fbfff700 1 --2- 172.21.15.150:0/3950059982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc102ab0 0x7f88fc0767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:06.165 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.166+0000 7f88fbfff700 1 -- 172.21.15.150:0/3950059982 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88fc075c20 con 0x7f88fc104650 2024-04-25T12:02:06.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.167+0000 7f88fbfff700 1 --2- 172.21.15.150:0/3950059982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88fc104650 0x7f88fc075370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f88f400ada0 tx=0x7f88f400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:02:06.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.167+0000 7f88f8ff9700 1 -- 172.21.15.150:0/3950059982 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4017020 con 0x7f88fc104650 2024-04-25T12:02:06.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.167+0000 7f88f8ff9700 1 -- 172.21.15.150:0/3950059982 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f88f400cda0 con 0x7f88fc104650 2024-04-25T12:02:06.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.167+0000 7f88f8ff9700 1 -- 172.21.15.150:0/3950059982 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f4005310 con 0x7f88fc104650 2024-04-25T12:02:06.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.167+0000 7f8901b68700 1 -- 172.21.15.150:0/3950059982 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88fc1a2eb0 con 0x7f88fc104650 2024-04-25T12:02:06.166 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.167+0000 7f8901b68700 1 -- 172.21.15.150:0/3950059982 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88fc1a3310 con 0x7f88fc104650 2024-04-25T12:02:06.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.168+0000 7f8901b68700 1 -- 172.21.15.150:0/3950059982 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88fc069730 con 0x7f88fc104650 2024-04-25T12:02:06.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.169+0000 7f88f8ff9700 1 -- 172.21.15.150:0/3950059982 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f88f400c8d0 con 0x7f88fc104650 2024-04-25T12:02:06.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.169+0000 7f88f8ff9700 1 --2- 172.21.15.150:0/3950059982 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88e80a3cc0 0x7f88e80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:02:06.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.170+0000 7f88f8ff9700 1 -- 172.21.15.150:0/3950059982 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f88f40cf3a0 con 0x7f88fc104650 2024-04-25T12:02:06.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.170+0000 7f88fb7fe700 1 --2- 172.21.15.150:0/3950059982 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88e80a3cc0 0x7f88e80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:02:06.167 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.170+0000 7f88fb7fe700 1 --2- 172.21.15.150:0/3950059982 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88e80a3cc0 0x7f88e80a6180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f88fc103b10 tx=0x7f88e4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:02:06.169 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.173+0000 7f88f8ff9700 1 -- 172.21.15.150:0/3950059982 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88f4021020 con 0x7f88fc104650 2024-04-25T12:02:06.374 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:06.378+0000 7f8901b68700 1 -- 172.21.15.150:0/3950059982 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f88fc062e80 con 0x7f88e80a3cc0 2024-04-25T12:02:06.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:06 smithi150 ceph-mon[32250]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:06.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:06 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:02:06.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:06 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:02:06.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:06 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:06.805 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:06 smithi169 ceph-mon[32710]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:06.805 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:06 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:02:06.805 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:06 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:02:06.805 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:06 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:07.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:06 smithi019 ceph-mon[29213]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:07.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:06 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:02:07.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:06 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:02:07.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:06 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:07.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:07 smithi150 ceph-mon[32250]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:02:07.903 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:07 smithi169 ceph-mon[32710]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:02:08.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:07 smithi019 ceph-mon[29213]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:02:08.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:08 smithi150 ceph-mon[32250]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:08.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:08 smithi169 ceph-mon[32710]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:09.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:08 smithi019 ceph-mon[29213]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:09.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:09 smithi169 ceph-mon[32710]: from='client.? 172.21.15.150:0/3369068022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19b7274-6545-43ee-891a-30f25c2c2e41"}]: dispatch 2024-04-25T12:02:09.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:09 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19b7274-6545-43ee-891a-30f25c2c2e41"}]: dispatch 2024-04-25T12:02:09.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:09 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e19b7274-6545-43ee-891a-30f25c2c2e41"}]': finished 2024-04-25T12:02:09.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:09 smithi169 ceph-mon[32710]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T12:02:09.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:09 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:09.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:09 smithi150 ceph-mon[32250]: from='client.? 172.21.15.150:0/3369068022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19b7274-6545-43ee-891a-30f25c2c2e41"}]: dispatch 2024-04-25T12:02:09.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:09 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19b7274-6545-43ee-891a-30f25c2c2e41"}]: dispatch 2024-04-25T12:02:09.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:09 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e19b7274-6545-43ee-891a-30f25c2c2e41"}]': finished 2024-04-25T12:02:09.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:09 smithi150 ceph-mon[32250]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T12:02:09.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:09 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:10.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:09 smithi019 ceph-mon[29213]: from='client.? 172.21.15.150:0/3369068022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19b7274-6545-43ee-891a-30f25c2c2e41"}]: dispatch 2024-04-25T12:02:10.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:09 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19b7274-6545-43ee-891a-30f25c2c2e41"}]: dispatch 2024-04-25T12:02:10.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:09 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e19b7274-6545-43ee-891a-30f25c2c2e41"}]': finished 2024-04-25T12:02:10.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:09 smithi019 ceph-mon[29213]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T12:02:10.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:09 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:10.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:10 smithi169 ceph-mon[32710]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:10.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:10 smithi169 ceph-mon[32710]: from='client.? 172.21.15.150:0/130201240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:02:10.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:10 smithi150 ceph-mon[32250]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:10.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:10 smithi150 ceph-mon[32250]: from='client.? 172.21.15.150:0/130201240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:02:11.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:10 smithi019 ceph-mon[29213]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:11.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:10 smithi019 ceph-mon[29213]: from='client.? 172.21.15.150:0/130201240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:02:12.849 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:12 smithi150 ceph-mon[32250]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:12.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:12 smithi169 ceph-mon[32710]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:13.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:12 smithi019 ceph-mon[29213]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:14.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:14 smithi169 ceph-mon[32710]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:14.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:14 smithi150 ceph-mon[32250]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:15.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:14 smithi019 ceph-mon[29213]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:16.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:16 smithi150 ceph-mon[32250]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:16.847 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:16 smithi019 ceph-mon[29213]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:16.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:16 smithi169 ceph-mon[32710]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:17.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:17 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T12:02:17.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:17 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:17.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:17 smithi169 ceph-mon[32710]: Deploying daemon osd.7 on smithi150 2024-04-25T12:02:17.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:17 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T12:02:17.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:17 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:17.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:17 smithi150 ceph-mon[32250]: Deploying daemon osd.7 on smithi150 2024-04-25T12:02:18.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:17 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T12:02:18.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:17 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:18.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:17 smithi019 ceph-mon[29213]: Deploying daemon osd.7 on smithi150 2024-04-25T12:02:18.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:18 smithi150 ceph-mon[32250]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:18.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:18 smithi169 ceph-mon[32710]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:19.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:18 smithi019 ceph-mon[29213]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:20.765 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:20 smithi150 ceph-mon[32250]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:20.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:20 smithi169 ceph-mon[32710]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:21.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:20 smithi019 ceph-mon[29213]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:22.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:22 smithi169 ceph-mon[32710]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:22.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:22 smithi150 ceph-mon[32250]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:23.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:22 smithi019 ceph-mon[29213]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:23.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:23.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:23.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:23.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:23.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:23.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:24.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:24.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:24.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:24.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:24 smithi169 ceph-mon[32710]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:24.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:24 smithi150 ceph-mon[32250]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:25.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:24 smithi019 ceph-mon[29213]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:26.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:26.599+0000 7f88f8ff9700 1 -- 172.21.15.150:0/3950059982 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f88fc062e80 con 0x7f88e80a3cc0 2024-04-25T12:02:26.596 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 7 on host 'smithi150' 2024-04-25T12:02:26.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:26.601+0000 7f8901b68700 1 -- 172.21.15.150:0/3950059982 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88e80a3cc0 msgr2=0x7f88e80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:26.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:26.601+0000 7f8901b68700 1 --2- 172.21.15.150:0/3950059982 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88e80a3cc0 0x7f88e80a6180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f88fc103b10 tx=0x7f88e4009190 comp rx=0 tx=0).stop 2024-04-25T12:02:26.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:26.601+0000 7f8901b68700 1 -- 172.21.15.150:0/3950059982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88fc104650 msgr2=0x7f88fc075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:26.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:26.601+0000 7f8901b68700 1 --2- 172.21.15.150:0/3950059982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88fc104650 0x7f88fc075370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f88f400ada0 tx=0x7f88f400c330 comp rx=0 tx=0).stop 2024-04-25T12:02:26.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:26.601+0000 7f8901b68700 1 -- 172.21.15.150:0/3950059982 shutdown_connections 2024-04-25T12:02:26.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:26.601+0000 7f8901b68700 1 --2- 172.21.15.150:0/3950059982 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88e80a3cc0 0x7f88e80a6180 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:26.598 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:26.601+0000 7f8901b68700 1 --2- 172.21.15.150:0/3950059982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88fc102ab0 0x7f88fc0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:26.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:26.601+0000 7f8901b68700 1 --2- 172.21.15.150:0/3950059982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88fc104650 0x7f88fc075370 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:26.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:26.601+0000 7f8901b68700 1 --2- 172.21.15.150:0/3950059982 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88fc103cb0 0x7f88fc074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:26.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:26.601+0000 7f8901b68700 1 -- 172.21.15.150:0/3950059982 >> 172.21.15.150:0/3950059982 conn(0x7f88fc0fe050 msgr2=0x7f88fc106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:02:26.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:26.602+0000 7f8901b68700 1 -- 172.21.15.150:0/3950059982 shutdown_connections 2024-04-25T12:02:26.599 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:02:26.602+0000 7f8901b68700 1 -- 172.21.15.150:0/3950059982 wait complete. 2024-04-25T12:02:26.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:26 smithi169 ceph-mon[32710]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:26.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:26.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:26.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:26.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:26.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:26.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:26 smithi169 ceph-mon[32710]: from='osd.7 [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T12:02:26.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:26 smithi169 ceph-mon[32710]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T12:02:26.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:26.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:26.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:26.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:26 smithi150 ceph-mon[32250]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:26.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:26.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:26.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:26.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:26.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:26.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:26 smithi150 ceph-mon[32250]: from='osd.7 [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T12:02:26.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:26 smithi150 ceph-mon[32250]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T12:02:26.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:26.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:26.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:27.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:26 smithi019 ceph-mon[29213]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:27.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:27.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:27.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:27.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:27.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:27.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:26 smithi019 ceph-mon[29213]: from='osd.7 [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T12:02:27.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:26 smithi019 ceph-mon[29213]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T12:02:27.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:27.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:27.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:27.331 DEBUG:teuthology.orchestra.run.smithi150:osd.7> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T12:02:27.333 INFO:tasks.cephadm:Deploying osd.8 on smithi169 with /dev/vg_nvme/lv_2... 2024-04-25T12:02:27.333 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T12:02:27.582 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:02:27.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:27 smithi150 ceph-mon[32250]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T12:02:27.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:27 smithi150 ceph-mon[32250]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T12:02:27.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:27 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:27.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:27 smithi150 ceph-mon[32250]: from='osd.7 [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:02:27.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:27 smithi150 ceph-mon[32250]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:02:27.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:27 smithi150 ceph-mon[32250]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:27 smithi169 ceph-mon[32710]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T12:02:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:27 smithi169 ceph-mon[32710]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T12:02:27.980 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:27 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:27 smithi169 ceph-mon[32710]: from='osd.7 [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:02:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:27 smithi169 ceph-mon[32710]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:02:27.981 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:27 smithi169 ceph-mon[32710]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:28.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:27 smithi019 ceph-mon[29213]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T12:02:28.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:27 smithi019 ceph-mon[29213]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T12:02:28.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:27 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:28.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:27 smithi019 ceph-mon[29213]: from='osd.7 [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:02:28.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:27 smithi019 ceph-mon[29213]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:02:28.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:27 smithi019 ceph-mon[29213]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:28.522 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:02:28 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:02:28.179+0000 7fb4e9a34700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T12:02:28.522 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:02:28 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:02:28.188+0000 7fb4e181f700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:02:29.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:28 smithi019 ceph-mon[29213]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T12:02:29.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:28 smithi019 ceph-mon[29213]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T12:02:29.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:28 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:29.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:28 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:29.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:28 smithi019 ceph-mon[29213]: from='osd.7 ' entity='osd.7' 2024-04-25T12:02:29.096 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:28 smithi169 ceph-mon[32710]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T12:02:29.096 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:28 smithi169 ceph-mon[32710]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T12:02:29.096 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:28 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:29.096 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:28 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:29.096 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:28 smithi169 ceph-mon[32710]: from='osd.7 ' entity='osd.7' 2024-04-25T12:02:29.113 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:28 smithi150 ceph-mon[32250]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T12:02:29.113 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:28 smithi150 ceph-mon[32250]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T12:02:29.114 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:28 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:29.114 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:28 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:29.114 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:28 smithi150 ceph-mon[32250]: from='osd.7 ' entity='osd.7' 2024-04-25T12:02:29.647 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T12:02:29.647 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T12:02:29.647 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T12:02:29.647 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T12:02:29.647 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115419 s, 908 MB/s 2024-04-25T12:02:29.647 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T12:02:30.009 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:29 smithi169 ceph-mon[32710]: purged_snaps scrub starts 2024-04-25T12:02:30.009 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:29 smithi169 ceph-mon[32710]: purged_snaps scrub ok 2024-04-25T12:02:30.009 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:30.009 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:29 smithi169 ceph-mon[32710]: osd.7 [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658] boot 2024-04-25T12:02:30.009 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:29 smithi169 ceph-mon[32710]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T12:02:30.009 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:30.009 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:29 smithi169 ceph-mon[32710]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:30.025 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:29 smithi150 ceph-mon[32250]: purged_snaps scrub starts 2024-04-25T12:02:30.026 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:29 smithi150 ceph-mon[32250]: purged_snaps scrub ok 2024-04-25T12:02:30.026 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:30.026 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:29 smithi150 ceph-mon[32250]: osd.7 [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658] boot 2024-04-25T12:02:30.026 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:29 smithi150 ceph-mon[32250]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T12:02:30.026 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:30.026 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:29 smithi150 ceph-mon[32250]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:30.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:29 smithi019 ceph-mon[29213]: purged_snaps scrub starts 2024-04-25T12:02:30.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:29 smithi019 ceph-mon[29213]: purged_snaps scrub ok 2024-04-25T12:02:30.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:30.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:29 smithi019 ceph-mon[29213]: osd.7 [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658] boot 2024-04-25T12:02:30.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:29 smithi019 ceph-mon[29213]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T12:02:30.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:02:30.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:29 smithi019 ceph-mon[29213]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T12:02:30.281 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_2 2024-04-25T12:02:30.523 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:02:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:30 smithi169 ceph-mon[32710]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T12:02:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:30 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:30 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:30 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:30 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:30 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:30 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:30 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:30 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:30.889 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:30 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:31.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:30 smithi019 ceph-mon[29213]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T12:02:31.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:30 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:31.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:30 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:31.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:30 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:31.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:30 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:31.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:30 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:31.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:30 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:31.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:30 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:31.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:30 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:31.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:30 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:31.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:30 smithi150 ceph-mon[32250]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T12:02:31.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:30 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:31.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:30 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:31.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:30 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:31.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:30 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:31.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:30 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:31.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:30 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:31.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:30 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:31.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:30 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:31.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:30 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:32.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.021+0000 7f71ad18c700 1 -- 172.21.15.169:0/2615298335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a8105ea0 msgr2=0x7f71a8108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:32.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.021+0000 7f71ad18c700 1 --2- 172.21.15.169:0/2615298335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a8105ea0 0x7f71a8108290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7190009990 tx=0x7f7190009ca0 comp rx=0 tx=0).stop 2024-04-25T12:02:32.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.021+0000 7f71ad18c700 1 -- 172.21.15.169:0/2615298335 shutdown_connections 2024-04-25T12:02:32.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.021+0000 7f71ad18c700 1 --2- 172.21.15.169:0/2615298335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a81087d0 0x7f71a810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:32.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.021+0000 7f71ad18c700 1 --2- 172.21.15.169:0/2615298335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a8103570 0x7f71a8105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:32.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.021+0000 7f71ad18c700 1 --2- 172.21.15.169:0/2615298335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a8105ea0 0x7f71a8108290 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:32.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.021+0000 7f71ad18c700 1 -- 172.21.15.169:0/2615298335 >> 172.21.15.169:0/2615298335 conn(0x7f71a80fcfa0 msgr2=0x7f71a80ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:02:32.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.021+0000 7f71ad18c700 1 -- 172.21.15.169:0/2615298335 shutdown_connections 2024-04-25T12:02:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.021+0000 7f71ad18c700 1 -- 172.21.15.169:0/2615298335 wait complete. 2024-04-25T12:02:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f71ad18c700 1 Processor -- start 2024-04-25T12:02:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f71ad18c700 1 -- start start 2024-04-25T12:02:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f71ad18c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a8103570 0x7f71a8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:02:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f71ad18c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a8105ea0 0x7f71a81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:02:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f71ad18c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a81087d0 0x7f71a81a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:02:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f71ad18c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f71a810e6a0 con 0x7f71a8103570 2024-04-25T12:02:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f71ad18c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f71a810e520 con 0x7f71a81087d0 2024-04-25T12:02:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f71ad18c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f71a810e820 con 0x7f71a8105ea0 2024-04-25T12:02:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f719ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a8105ea0 0x7f71a81991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:02:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f719ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a8105ea0 0x7f71a81991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:59004/0 (socket says 172.21.15.169:59004) 2024-04-25T12:02:32.021 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f719ffff700 1 -- 172.21.15.169:0/3693090151 learned_addr learned my addr 172.21.15.169:0/3693090151 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T12:02:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f71a759e700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a81087d0 0x7f71a81a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:02:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f71a6d9d700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a8103570 0x7f71a8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:02:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f719ffff700 1 -- 172.21.15.169:0/3693090151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a81087d0 msgr2=0x7f71a81a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f719ffff700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a81087d0 0x7f71a81a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f719ffff700 1 -- 172.21.15.169:0/3693090151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a8103570 msgr2=0x7f71a8198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f719ffff700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a8103570 0x7f71a8198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f719ffff700 1 -- 172.21.15.169:0/3693090151 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a81a08c0 con 0x7f71a8105ea0 2024-04-25T12:02:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f71a759e700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a81087d0 0x7f71a81a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:02:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.022+0000 7f719ffff700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a8105ea0 0x7f71a81991b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7190006010 tx=0x7f7190009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:02:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.023+0000 7f71a4d99700 1 -- 172.21.15.169:0/3693090151 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190027070 con 0x7f71a8105ea0 2024-04-25T12:02:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.023+0000 7f71ad18c700 1 -- 172.21.15.169:0/3693090151 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a81a0b50 con 0x7f71a8105ea0 2024-04-25T12:02:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.023+0000 7f71a4d99700 1 -- 172.21.15.169:0/3693090151 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7190004110 con 0x7f71a8105ea0 2024-04-25T12:02:32.022 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.023+0000 7f71a4d99700 1 -- 172.21.15.169:0/3693090151 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719000f460 con 0x7f71a8105ea0 2024-04-25T12:02:32.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.023+0000 7f71ad18c700 1 -- 172.21.15.169:0/3693090151 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a81a1000 con 0x7f71a8105ea0 2024-04-25T12:02:32.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.024+0000 7f71ad18c700 1 -- 172.21.15.169:0/3693090151 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a8069730 con 0x7f71a8105ea0 2024-04-25T12:02:32.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.025+0000 7f71a4d99700 1 -- 172.21.15.169:0/3693090151 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f71900226b0 con 0x7f71a8105ea0 2024-04-25T12:02:32.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.025+0000 7f71a4d99700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f71940a3c40 0x7f71940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:02:32.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.025+0000 7f71a4d99700 1 -- 172.21.15.169:0/3693090151 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7f71900d0640 con 0x7f71a8105ea0 2024-04-25T12:02:32.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.026+0000 7f71a6d9d700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f71940a3c40 0x7f71940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:02:32.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.026+0000 7f71a6d9d700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f71940a3c40 0x7f71940a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7198005ea0 tx=0x7f7198005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:02:32.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.030+0000 7f71a4d99700 1 -- 172.21.15.169:0/3693090151 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7190013070 con 0x7f71a8105ea0 2024-04-25T12:02:32.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:31 smithi169 ceph-mon[32710]: Detected new or changed devices on smithi150 2024-04-25T12:02:32.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:31 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi150 to 5696M 2024-04-25T12:02:32.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:31 smithi169 ceph-mon[32710]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T12:02:32.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:31 smithi169 ceph-mon[32710]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:32.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:31 smithi150 ceph-mon[32250]: Detected new or changed devices on smithi150 2024-04-25T12:02:32.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:31 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi150 to 5696M 2024-04-25T12:02:32.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:31 smithi150 ceph-mon[32250]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T12:02:32.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:31 smithi150 ceph-mon[32250]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:32.232 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:32.236+0000 7f71ad18c700 1 -- 172.21.15.169:0/3693090151 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f71a8062e80 con 0x7f71940a3c40 2024-04-25T12:02:32.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:31 smithi019 ceph-mon[29213]: Detected new or changed devices on smithi150 2024-04-25T12:02:32.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:31 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi150 to 5696M 2024-04-25T12:02:32.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:31 smithi019 ceph-mon[29213]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T12:02:32.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:31 smithi019 ceph-mon[29213]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:33.074 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:32 smithi169 ceph-mon[32710]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T12:02:33.074 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:32 smithi169 ceph-mon[32710]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:02:33.075 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:32 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:02:33.075 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:32 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:02:33.075 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:32 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:33.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:32 smithi150 ceph-mon[32250]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T12:02:33.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:32 smithi150 ceph-mon[32250]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:02:33.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:32 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:02:33.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:32 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:02:33.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:32 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:33.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:32 smithi019 ceph-mon[29213]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T12:02:33.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:32 smithi019 ceph-mon[29213]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:02:33.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:32 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:02:33.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:32 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:02:33.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:32 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:34.113 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:33 smithi169 ceph-mon[32710]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T12:02:34.113 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:33 smithi169 ceph-mon[32710]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:34.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:33 smithi150 ceph-mon[32250]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T12:02:34.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:33 smithi150 ceph-mon[32250]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:34.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:33 smithi019 ceph-mon[29213]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T12:02:34.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:33 smithi019 ceph-mon[29213]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:36.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:35 smithi019 ceph-mon[29213]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:36.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:35 smithi019 ceph-mon[29213]: from='client.? 172.21.15.169:0/316734141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8175ec26-2033-4312-8778-2d2e8512e4cf"}]: dispatch 2024-04-25T12:02:36.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:35 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8175ec26-2033-4312-8778-2d2e8512e4cf"}]: dispatch 2024-04-25T12:02:36.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:35 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8175ec26-2033-4312-8778-2d2e8512e4cf"}]': finished 2024-04-25T12:02:36.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:35 smithi019 ceph-mon[29213]: osdmap e56: 9 total, 8 up, 9 in 2024-04-25T12:02:36.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:36.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:35 smithi019 ceph-mon[29213]: from='client.? 172.21.15.169:0/3847228307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:02:36.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:35 smithi169 ceph-mon[32710]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:36.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:35 smithi169 ceph-mon[32710]: from='client.? 172.21.15.169:0/316734141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8175ec26-2033-4312-8778-2d2e8512e4cf"}]: dispatch 2024-04-25T12:02:36.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:35 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8175ec26-2033-4312-8778-2d2e8512e4cf"}]: dispatch 2024-04-25T12:02:36.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:35 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8175ec26-2033-4312-8778-2d2e8512e4cf"}]': finished 2024-04-25T12:02:36.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:35 smithi169 ceph-mon[32710]: osdmap e56: 9 total, 8 up, 9 in 2024-04-25T12:02:36.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:35 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:36.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:35 smithi169 ceph-mon[32710]: from='client.? 172.21.15.169:0/3847228307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:02:36.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:35 smithi150 ceph-mon[32250]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:36.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:35 smithi150 ceph-mon[32250]: from='client.? 172.21.15.169:0/316734141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8175ec26-2033-4312-8778-2d2e8512e4cf"}]: dispatch 2024-04-25T12:02:36.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:35 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8175ec26-2033-4312-8778-2d2e8512e4cf"}]: dispatch 2024-04-25T12:02:36.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:35 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8175ec26-2033-4312-8778-2d2e8512e4cf"}]': finished 2024-04-25T12:02:36.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:35 smithi150 ceph-mon[32250]: osdmap e56: 9 total, 8 up, 9 in 2024-04-25T12:02:36.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:35 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:36.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:35 smithi150 ceph-mon[32250]: from='client.? 172.21.15.169:0/3847228307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:02:38.214 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:37 smithi150 ceph-mon[32250]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:38.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:37 smithi019 ceph-mon[29213]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:38.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:37 smithi169 ceph-mon[32710]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:40.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:39 smithi169 ceph-mon[32710]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:40.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:39 smithi019 ceph-mon[29213]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:40.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:39 smithi150 ceph-mon[32250]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:42.178 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:41 smithi169 ceph-mon[32710]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:42.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:41 smithi019 ceph-mon[29213]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:42.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:41 smithi150 ceph-mon[32250]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:43.111 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:42 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T12:02:43.111 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:42 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:43.111 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:42 smithi169 ceph-mon[32710]: Deploying daemon osd.8 on smithi169 2024-04-25T12:02:43.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:42 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T12:02:43.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:42 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:43.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:42 smithi019 ceph-mon[29213]: Deploying daemon osd.8 on smithi169 2024-04-25T12:02:43.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:42 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T12:02:43.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:42 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:43.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:42 smithi150 ceph-mon[32250]: Deploying daemon osd.8 on smithi169 2024-04-25T12:02:44.137 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:43 smithi169 ceph-mon[32710]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:44.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:43 smithi019 ceph-mon[29213]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:44.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:43 smithi150 ceph-mon[32250]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:46.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:45 smithi169 ceph-mon[32710]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:46.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:45 smithi019 ceph-mon[29213]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:46.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:45 smithi150 ceph-mon[32250]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:48.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:48 smithi169 ceph-mon[32710]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:48.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:48 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:48.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:48 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:48.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:48 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:49.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:48 smithi019 ceph-mon[29213]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:49.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:48 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:49.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:48 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:49.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:48 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:49.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:48 smithi150 ceph-mon[32250]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:49.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:48 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:49.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:48 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:49.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:48 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:51.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:50 smithi019 ceph-mon[29213]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:51.077 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:50 smithi169 ceph-mon[32710]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:51.149 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:51.153+0000 7f71a4d99700 1 -- 172.21.15.169:0/3693090151 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f71a8062e80 con 0x7f71940a3c40 2024-04-25T12:02:51.150 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 8 on host 'smithi169' 2024-04-25T12:02:51.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:51.155+0000 7f71ad18c700 1 -- 172.21.15.169:0/3693090151 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f71940a3c40 msgr2=0x7f71940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:51.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:51.155+0000 7f71ad18c700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f71940a3c40 0x7f71940a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7198005ea0 tx=0x7f7198005e30 comp rx=0 tx=0).stop 2024-04-25T12:02:51.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:51.155+0000 7f71ad18c700 1 -- 172.21.15.169:0/3693090151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a8105ea0 msgr2=0x7f71a81991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:51.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:51.155+0000 7f71ad18c700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a8105ea0 0x7f71a81991b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7190006010 tx=0x7f7190009f90 comp rx=0 tx=0).stop 2024-04-25T12:02:51.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:51.155+0000 7f71ad18c700 1 -- 172.21.15.169:0/3693090151 shutdown_connections 2024-04-25T12:02:51.152 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:51.155+0000 7f71ad18c700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f71940a3c40 0x7f71940a6100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:51.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:51.155+0000 7f71ad18c700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71a8103570 0x7f71a8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:51.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:51.155+0000 7f71ad18c700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71a81087d0 0x7f71a81a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:51.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:51.155+0000 7f71ad18c700 1 --2- 172.21.15.169:0/3693090151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f71a8105ea0 0x7f71a81991b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:51.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:51.155+0000 7f71ad18c700 1 -- 172.21.15.169:0/3693090151 >> 172.21.15.169:0/3693090151 conn(0x7f71a80fcfa0 msgr2=0x7f71a8103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:02:51.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:51.155+0000 7f71ad18c700 1 -- 172.21.15.169:0/3693090151 shutdown_connections 2024-04-25T12:02:51.153 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:02:51.155+0000 7f71ad18c700 1 -- 172.21.15.169:0/3693090151 wait complete. 2024-04-25T12:02:51.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:50 smithi150 ceph-mon[32250]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:51.767 DEBUG:teuthology.orchestra.run.smithi169:osd.8> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T12:02:51.769 INFO:tasks.cephadm:Deploying osd.9 on smithi019 with /dev/vg_nvme/lv_1... 2024-04-25T12:02:51.770 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T12:02:52.031 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:02:52.390 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:52 smithi019 ceph-mon[29213]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:52.390 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:52 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:52.391 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:52 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:52.391 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:52 smithi019 ceph-mon[29213]: from='osd.8 [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T12:02:52.391 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:52 smithi019 ceph-mon[29213]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T12:02:52.391 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:52 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:52.391 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:52 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:52.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:52 smithi169 ceph-mon[32710]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:52.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:52 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:52.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:52 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:52.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:52 smithi169 ceph-mon[32710]: from='osd.8 [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T12:02:52.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:52 smithi169 ceph-mon[32710]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T12:02:52.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:52 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:52.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:52 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:52.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:52 smithi150 ceph-mon[32250]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:52.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:52 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:52.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:52 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:52.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:52 smithi150 ceph-mon[32250]: from='osd.8 [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T12:02:52.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:52 smithi150 ceph-mon[32250]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T12:02:52.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:52 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:52.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:52 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:53.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:53 smithi169 ceph-mon[32710]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T12:02:53.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:53 smithi169 ceph-mon[32710]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T12:02:53.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:53 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:53.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:53 smithi169 ceph-mon[32710]: from='osd.8 [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:02:53.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:53 smithi169 ceph-mon[32710]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:02:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:53 smithi019 ceph-mon[29213]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T12:02:53.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:53 smithi019 ceph-mon[29213]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T12:02:53.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:53 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:53.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:53 smithi019 ceph-mon[29213]: from='osd.8 [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:02:53.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:53 smithi019 ceph-mon[29213]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:02:53.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:53 smithi150 ceph-mon[32250]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T12:02:53.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:53 smithi150 ceph-mon[32250]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T12:02:53.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:53 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:53.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:53 smithi150 ceph-mon[32250]: from='osd.8 [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:02:53.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:53 smithi150 ceph-mon[32250]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:02:53.876 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:02:53 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:02:53.533+0000 7f78a7f75700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T12:02:53.876 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:02:53 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:02:53.544+0000 7f789fd60700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:02:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T12:02:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T12:02:54.126 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-04-25T12:02:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-04-25T12:02:54.126 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013708 s, 765 MB/s 2024-04-25T12:02:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-04-25T12:02:54.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:54 smithi169 ceph-mon[32710]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:54.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:54 smithi169 ceph-mon[32710]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T12:02:54.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:54 smithi169 ceph-mon[32710]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T12:02:54.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:54.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:54.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:54 smithi169 ceph-mon[32710]: from='osd.8 ' entity='osd.8' 2024-04-25T12:02:54.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:54 smithi150 ceph-mon[32250]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:54.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:54 smithi150 ceph-mon[32250]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T12:02:54.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:54 smithi150 ceph-mon[32250]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T12:02:54.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:54.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:54.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:54 smithi150 ceph-mon[32250]: from='osd.8 ' entity='osd.8' 2024-04-25T12:02:54.436 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:54 smithi019 ceph-mon[29213]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:54.436 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:54 smithi019 ceph-mon[29213]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T12:02:54.436 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:54 smithi019 ceph-mon[29213]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T12:02:54.436 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:54.436 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:54.436 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:54 smithi019 ceph-mon[29213]: from='osd.8 ' entity='osd.8' 2024-04-25T12:02:54.791 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_1 2024-04-25T12:02:55.042 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:02:55.373 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:55 smithi169 ceph-mon[32710]: purged_snaps scrub starts 2024-04-25T12:02:55.373 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:55 smithi169 ceph-mon[32710]: purged_snaps scrub ok 2024-04-25T12:02:55.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:55.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:55 smithi169 ceph-mon[32710]: osd.8 [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254] boot 2024-04-25T12:02:55.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:55 smithi169 ceph-mon[32710]: osdmap e59: 9 total, 9 up, 9 in 2024-04-25T12:02:55.374 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:55.411 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:55 smithi150 ceph-mon[32250]: purged_snaps scrub starts 2024-04-25T12:02:55.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:55 smithi150 ceph-mon[32250]: purged_snaps scrub ok 2024-04-25T12:02:55.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:55.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:55 smithi150 ceph-mon[32250]: osd.8 [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254] boot 2024-04-25T12:02:55.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:55 smithi150 ceph-mon[32250]: osdmap e59: 9 total, 9 up, 9 in 2024-04-25T12:02:55.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:55.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:55 smithi019 ceph-mon[29213]: purged_snaps scrub starts 2024-04-25T12:02:55.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:55 smithi019 ceph-mon[29213]: purged_snaps scrub ok 2024-04-25T12:02:55.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:55.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:55 smithi019 ceph-mon[29213]: osd.8 [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254] boot 2024-04-25T12:02:55.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:55 smithi019 ceph-mon[29213]: osdmap e59: 9 total, 9 up, 9 in 2024-04-25T12:02:55.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:02:56.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:56.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T12:02:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: Detected new or changed devices on smithi169 2024-04-25T12:02:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-25T12:02:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:56.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T12:02:56.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: Detected new or changed devices on smithi169 2024-04-25T12:02:56.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:56.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:56.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:56.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-25T12:02:56.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:56.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:56.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:56.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:56.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:56.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.447 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T12:02:56.447 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T12:02:56.447 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: Detected new or changed devices on smithi169 2024-04-25T12:02:56.447 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.447 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:56.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:56.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:02:56.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi169 to 5696M 2024-04-25T12:02:56.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:56.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:56.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:02:56.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:56.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:02:56.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:02:56.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.837+0000 7f11de4d4700 1 -- 172.21.15.19:0/3514979531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11d8105e70 msgr2=0x7f11d8108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:56.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.837+0000 7f11de4d4700 1 --2- 172.21.15.19:0/3514979531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11d8105e70 0x7f11d8108260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009990 tx=0x7f11cc009ca0 comp rx=0 tx=0).stop 2024-04-25T12:02:56.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.837+0000 7f11de4d4700 1 -- 172.21.15.19:0/3514979531 shutdown_connections 2024-04-25T12:02:56.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.837+0000 7f11de4d4700 1 --2- 172.21.15.19:0/3514979531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11d81087a0 0x7f11d810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:56.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.837+0000 7f11de4d4700 1 --2- 172.21.15.19:0/3514979531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d8103540 0x7f11d8105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:56.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.837+0000 7f11de4d4700 1 --2- 172.21.15.19:0/3514979531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11d8105e70 0x7f11d8108260 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:56.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.837+0000 7f11de4d4700 1 -- 172.21.15.19:0/3514979531 >> 172.21.15.19:0/3514979531 conn(0x7f11d80fcf70 msgr2=0x7f11d80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:02:56.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.837+0000 7f11de4d4700 1 -- 172.21.15.19:0/3514979531 shutdown_connections 2024-04-25T12:02:56.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.837+0000 7f11de4d4700 1 -- 172.21.15.19:0/3514979531 wait complete. 2024-04-25T12:02:56.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.838+0000 7f11de4d4700 1 Processor -- start 2024-04-25T12:02:56.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.838+0000 7f11de4d4700 1 -- start start 2024-04-25T12:02:56.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.838+0000 7f11de4d4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d8103540 0x7f11d8198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:02:56.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.838+0000 7f11de4d4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11d8105e70 0x7f11d8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:02:56.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.838+0000 7f11de4d4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11d81087a0 0x7f11d81a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:02:56.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.838+0000 7f11de4d4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11d810e690 con 0x7f11d8105e70 2024-04-25T12:02:56.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.838+0000 7f11de4d4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f11d810e510 con 0x7f11d81087a0 2024-04-25T12:02:56.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.838+0000 7f11de4d4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f11d810e810 con 0x7f11d8103540 2024-04-25T12:02:56.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.838+0000 7f11d77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11d8105e70 0x7f11d8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:02:56.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.838+0000 7f11d77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11d8105e70 0x7f11d8199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50978/0 (socket says 172.21.15.19:50978) 2024-04-25T12:02:56.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.838+0000 7f11d77fe700 1 -- 172.21.15.19:0/2116042795 learned_addr learned my addr 172.21.15.19:0/2116042795 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:02:56.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.838+0000 7f11d7fff700 1 --2- 172.21.15.19:0/2116042795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d8103540 0x7f11d8198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:02:56.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.838+0000 7f11dca71700 1 --2- 172.21.15.19:0/2116042795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11d81087a0 0x7f11d81a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:02:56.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.839+0000 7f11d77fe700 1 -- 172.21.15.19:0/2116042795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d8103540 msgr2=0x7f11d8198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:56.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.839+0000 7f11d77fe700 1 --2- 172.21.15.19:0/2116042795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d8103540 0x7f11d8198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:56.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.839+0000 7f11d77fe700 1 -- 172.21.15.19:0/2116042795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11d81087a0 msgr2=0x7f11d81a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:02:56.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.839+0000 7f11d77fe700 1 --2- 172.21.15.19:0/2116042795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11d81087a0 0x7f11d81a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:02:56.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.839+0000 7f11d77fe700 1 -- 172.21.15.19:0/2116042795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d81a0980 con 0x7f11d8105e70 2024-04-25T12:02:56.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.839+0000 7f11d77fe700 1 --2- 172.21.15.19:0/2116042795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11d8105e70 0x7f11d8199270 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00b2e0 tx=0x7f11cc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:02:56.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.839+0000 7f11d57fa700 1 -- 172.21.15.19:0/2116042795 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc027070 con 0x7f11d8105e70 2024-04-25T12:02:56.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.839+0000 7f11d57fa700 1 -- 172.21.15.19:0/2116042795 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f11cc00be70 con 0x7f11d8105e70 2024-04-25T12:02:56.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.839+0000 7f11de4d4700 1 -- 172.21.15.19:0/2116042795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d81a0c10 con 0x7f11d8105e70 2024-04-25T12:02:56.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.839+0000 7f11d57fa700 1 -- 172.21.15.19:0/2116042795 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc017650 con 0x7f11d8105e70 2024-04-25T12:02:56.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.839+0000 7f11de4d4700 1 -- 172.21.15.19:0/2116042795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d81a10c0 con 0x7f11d8105e70 2024-04-25T12:02:56.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.840+0000 7f11de4d4700 1 -- 172.21.15.19:0/2116042795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d8069730 con 0x7f11d8105e70 2024-04-25T12:02:56.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.841+0000 7f11d57fa700 1 -- 172.21.15.19:0/2116042795 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f11cc02a070 con 0x7f11d8105e70 2024-04-25T12:02:56.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.842+0000 7f11d57fa700 1 --2- 172.21.15.19:0/2116042795 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f11c00a3cc0 0x7f11c00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:02:56.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.842+0000 7f11d7fff700 1 --2- 172.21.15.19:0/2116042795 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f11c00a3cc0 0x7f11c00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:02:56.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.842+0000 7f11d57fa700 1 -- 172.21.15.19:0/2116042795 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5858+0+0 (secure 0 0 0) 0x7f11cc0d0e20 con 0x7f11d8105e70 2024-04-25T12:02:56.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.842+0000 7f11d7fff700 1 --2- 172.21.15.19:0/2116042795 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f11c00a3cc0 0x7f11c00a6180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f11d819a210 tx=0x7f11c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:02:56.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:56.846+0000 7f11d57fa700 1 -- 172.21.15.19:0/2116042795 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11cc031070 con 0x7f11d8105e70 2024-04-25T12:02:57.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:02:57.065+0000 7f11de4d4700 1 -- 172.21.15.19:0/2116042795 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f11d8062e80 con 0x7f11c00a3cc0 2024-04-25T12:02:57.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:57 smithi150 ceph-mon[32250]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T12:02:57.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:02:57.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:02:57.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:57 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:57.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:57 smithi019 ceph-mon[29213]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T12:02:57.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:02:57.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:02:57.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:57 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:57.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:57 smithi169 ceph-mon[32710]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T12:02:57.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:02:57.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:02:57.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:57 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:02:58.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:58 smithi019 ceph-mon[29213]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:02:58.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:58 smithi019 ceph-mon[29213]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:02:58.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:58 smithi019 ceph-mon[29213]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T12:02:58.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:58 smithi169 ceph-mon[32710]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:02:58.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:58 smithi169 ceph-mon[32710]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:02:58.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:58 smithi169 ceph-mon[32710]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T12:02:58.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:58 smithi150 ceph-mon[32250]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:02:58.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:58 smithi150 ceph-mon[32250]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:02:58.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:58 smithi150 ceph-mon[32250]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T12:02:59.255 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:02:59 smithi019 ceph-mon[29213]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T12:02:59.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:02:59 smithi169 ceph-mon[32710]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T12:02:59.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:02:59 smithi150 ceph-mon[32250]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T12:03:00.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:00 smithi019 ceph-mon[29213]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:00.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:00 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1369831863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2941192e-f2a5-447c-8185-61f976b616db"}]: dispatch 2024-04-25T12:03:00.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:00 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1369831863' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2941192e-f2a5-447c-8185-61f976b616db"}]': finished 2024-04-25T12:03:00.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:00 smithi019 ceph-mon[29213]: osdmap e64: 10 total, 9 up, 10 in 2024-04-25T12:03:00.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:00.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:00 smithi169 ceph-mon[32710]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:00.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:00 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1369831863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2941192e-f2a5-447c-8185-61f976b616db"}]: dispatch 2024-04-25T12:03:00.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:00 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1369831863' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2941192e-f2a5-447c-8185-61f976b616db"}]': finished 2024-04-25T12:03:00.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:00 smithi169 ceph-mon[32710]: osdmap e64: 10 total, 9 up, 10 in 2024-04-25T12:03:00.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:00.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:00 smithi150 ceph-mon[32250]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:00.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:00 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1369831863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2941192e-f2a5-447c-8185-61f976b616db"}]: dispatch 2024-04-25T12:03:00.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:00 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1369831863' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2941192e-f2a5-447c-8185-61f976b616db"}]': finished 2024-04-25T12:03:00.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:00 smithi150 ceph-mon[32250]: osdmap e64: 10 total, 9 up, 10 in 2024-04-25T12:03:00.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:01.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:01 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3304755680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:03:01.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:01 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3304755680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:03:01.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:01 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3304755680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:03:02.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:02 smithi019 ceph-mon[29213]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-25T12:03:02.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:02 smithi169 ceph-mon[32710]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-25T12:03:02.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:02 smithi150 ceph-mon[32250]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-25T12:03:04.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:04 smithi019 ceph-mon[29213]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T12:03:04.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:04 smithi169 ceph-mon[32710]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T12:03:04.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:04 smithi150 ceph-mon[32250]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T12:03:06.456 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:06 smithi169 ceph-mon[32710]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T12:03:06.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:06 smithi019 ceph-mon[29213]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T12:03:06.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:06 smithi150 ceph-mon[32250]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T12:03:08.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:08 smithi150 ceph-mon[32250]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T12:03:08.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T12:03:08.464 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:08.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:08 smithi019 ceph-mon[29213]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T12:03:08.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T12:03:08.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:08.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:08 smithi169 ceph-mon[32710]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T12:03:08.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T12:03:08.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:09.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:09 smithi019 ceph-mon[29213]: Deploying daemon osd.9 on smithi019 2024-04-25T12:03:09.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:09 smithi169 ceph-mon[32710]: Deploying daemon osd.9 on smithi019 2024-04-25T12:03:09.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:09 smithi150 ceph-mon[32250]: Deploying daemon osd.9 on smithi019 2024-04-25T12:03:10.467 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:10 smithi019 ceph-mon[29213]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T12:03:10.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:10 smithi169 ceph-mon[32710]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T12:03:10.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:10 smithi150 ceph-mon[32250]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T12:03:12.233 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:12 smithi019 ceph-mon[29213]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:12.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:12 smithi169 ceph-mon[32710]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:12.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:12 smithi150 ceph-mon[32250]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:14.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:14 smithi019 ceph-mon[29213]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:14.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:14 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:14.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:14 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:03:14.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:14 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:14.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:14 smithi169 ceph-mon[32710]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:14.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:14 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:14.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:14 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:03:14.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:14 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:14.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:14 smithi150 ceph-mon[32250]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:14.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:14 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:14.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:14 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:03:14.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:14 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:16.505 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:16 smithi019 ceph-mon[29213]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:16.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:16 smithi169 ceph-mon[32710]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:16.661 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:16 smithi150 ceph-mon[32250]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:17.982 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 9 on host 'smithi019' 2024-04-25T12:03:17.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:03:17.985+0000 7f11d57fa700 1 -- 172.21.15.19:0/2116042795 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f11d8062e80 con 0x7f11c00a3cc0 2024-04-25T12:03:17.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:03:17.987+0000 7f11de4d4700 1 -- 172.21.15.19:0/2116042795 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f11c00a3cc0 msgr2=0x7f11c00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:03:17.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:03:17.987+0000 7f11de4d4700 1 --2- 172.21.15.19:0/2116042795 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f11c00a3cc0 0x7f11c00a6180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f11d819a210 tx=0x7f11c8009260 comp rx=0 tx=0).stop 2024-04-25T12:03:17.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:03:17.987+0000 7f11de4d4700 1 -- 172.21.15.19:0/2116042795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11d8105e70 msgr2=0x7f11d8199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:03:17.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:03:17.987+0000 7f11de4d4700 1 --2- 172.21.15.19:0/2116042795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11d8105e70 0x7f11d8199270 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00b2e0 tx=0x7f11cc00f690 comp rx=0 tx=0).stop 2024-04-25T12:03:17.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:03:17.988+0000 7f11de4d4700 1 -- 172.21.15.19:0/2116042795 shutdown_connections 2024-04-25T12:03:17.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:03:17.988+0000 7f11de4d4700 1 --2- 172.21.15.19:0/2116042795 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f11c00a3cc0 0x7f11c00a6180 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:17.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:03:17.988+0000 7f11de4d4700 1 --2- 172.21.15.19:0/2116042795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11d81087a0 0x7f11d81a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:17.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:03:17.988+0000 7f11de4d4700 1 --2- 172.21.15.19:0/2116042795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11d8103540 0x7f11d8198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:17.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:03:17.988+0000 7f11de4d4700 1 --2- 172.21.15.19:0/2116042795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11d8105e70 0x7f11d8199270 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:17.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:03:17.988+0000 7f11de4d4700 1 -- 172.21.15.19:0/2116042795 >> 172.21.15.19:0/2116042795 conn(0x7f11d80fcf70 msgr2=0x7f11d8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:03:17.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:03:17.988+0000 7f11de4d4700 1 -- 172.21.15.19:0/2116042795 shutdown_connections 2024-04-25T12:03:17.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:03:17.988+0000 7f11de4d4700 1 -- 172.21.15.19:0/2116042795 wait complete. 2024-04-25T12:03:18.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:18 smithi019 ceph-mon[29213]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:18.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:18.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:18 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:18.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:18 smithi019 ceph-mon[29213]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T12:03:18.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:18 smithi169 ceph-mon[32710]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:18.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:18 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:18.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:18 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:18.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:18 smithi169 ceph-mon[32710]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T12:03:18.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:18 smithi150 ceph-mon[32250]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:18.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:18 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:18.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:18 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:18.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:18 smithi150 ceph-mon[32250]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T12:03:18.817 DEBUG:teuthology.orchestra.run.smithi019:osd.9> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T12:03:18.819 INFO:tasks.cephadm:Deploying osd.10 on smithi150 with /dev/vg_nvme/lv_1... 2024-04-25T12:03:18.820 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T12:03:19.069 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:03:19.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:19 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:19.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:19 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:19.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:19 smithi150 ceph-mon[32250]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T12:03:19.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:19 smithi150 ceph-mon[32250]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T12:03:19.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:19 smithi150 ceph-mon[32250]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T12:03:19.463 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:19 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:19.654 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:19 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:19.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:19 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:19.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:19 smithi169 ceph-mon[32710]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T12:03:19.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:19 smithi169 ceph-mon[32710]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T12:03:19.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:19 smithi169 ceph-mon[32710]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T12:03:19.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:19 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:19.763 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:19 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:19.763 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:19 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:19.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:19 smithi019 ceph-mon[29213]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T12:03:19.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:19 smithi019 ceph-mon[29213]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T12:03:19.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:19 smithi019 ceph-mon[29213]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-04-25T12:03:19.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:19 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:20.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:20 smithi019 ceph-mon[29213]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:20.514 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:20 smithi019 ceph-mon[29213]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T12:03:20.514 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:20 smithi019 ceph-mon[29213]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T12:03:20.514 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:20 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:20.514 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:20 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:20.514 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:03:20 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:03:20.398+0000 7f2d55482700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T12:03:20.514 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:03:20 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:03:20.411+0000 7f2d4d26d700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:03:20.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:20 smithi150 ceph-mon[32250]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:20.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:20 smithi150 ceph-mon[32250]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T12:03:20.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:20 smithi150 ceph-mon[32250]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T12:03:20.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:20 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:20.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:20 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:20.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:20 smithi169 ceph-mon[32710]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:20.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:20 smithi169 ceph-mon[32710]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-04-25T12:03:20.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:20 smithi169 ceph-mon[32710]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T12:03:20.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:20 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:20.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:20 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:21.330 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T12:03:21.331 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T12:03:21.331 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-25T12:03:21.331 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-25T12:03:21.331 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125292 s, 837 MB/s 2024-04-25T12:03:21.331 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-25T12:03:21.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:21 smithi150 ceph-mon[32250]: purged_snaps scrub starts 2024-04-25T12:03:21.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:21 smithi150 ceph-mon[32250]: purged_snaps scrub ok 2024-04-25T12:03:21.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:21 smithi150 ceph-mon[32250]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' 2024-04-25T12:03:21.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:21 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:21.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:21 smithi019 ceph-mon[29213]: purged_snaps scrub starts 2024-04-25T12:03:21.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:21 smithi019 ceph-mon[29213]: purged_snaps scrub ok 2024-04-25T12:03:21.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:21 smithi019 ceph-mon[29213]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' 2024-04-25T12:03:21.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:21 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:21.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:21 smithi169 ceph-mon[32710]: purged_snaps scrub starts 2024-04-25T12:03:21.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:21 smithi169 ceph-mon[32710]: purged_snaps scrub ok 2024-04-25T12:03:21.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:21 smithi169 ceph-mon[32710]: from='osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664]' entity='osd.9' 2024-04-25T12:03:21.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:21 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:22.039 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_1 2024-04-25T12:03:22.288 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:03:22.404 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:22 smithi150 ceph-mon[32250]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:22.546 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:22 smithi019 ceph-mon[29213]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:22.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:22 smithi019 ceph-mon[29213]: osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664] boot 2024-04-25T12:03:22.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:22 smithi019 ceph-mon[29213]: osdmap e67: 10 total, 10 up, 10 in 2024-04-25T12:03:22.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:22.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:22.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:22.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:22.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:22.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:22.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:22.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:22.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:22.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:03:22.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:22.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:22 smithi150 ceph-mon[32250]: osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664] boot 2024-04-25T12:03:22.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:22 smithi150 ceph-mon[32250]: osdmap e67: 10 total, 10 up, 10 in 2024-04-25T12:03:22.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:22.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:22.656 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:22.657 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:22.657 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:22.657 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:22.657 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:22.657 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:22.657 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:22.658 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:03:22.658 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:22.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:22 smithi169 ceph-mon[32710]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T12:03:22.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:22 smithi169 ceph-mon[32710]: osd.9 [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664] boot 2024-04-25T12:03:22.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:22 smithi169 ceph-mon[32710]: osdmap e67: 10 total, 10 up, 10 in 2024-04-25T12:03:22.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:03:22.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:22.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:22.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:22.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:22.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:22.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:22.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:22.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:22.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:03:22.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:23.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:23 smithi150 ceph-mon[32250]: Detected new or changed devices on smithi019 2024-04-25T12:03:23.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:23 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi019 to 4273M 2024-04-25T12:03:23.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:23 smithi150 ceph-mon[32250]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T12:03:23.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:03:23.663 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:23.663 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:03:23.663 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:23.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:23 smithi019 ceph-mon[29213]: Detected new or changed devices on smithi019 2024-04-25T12:03:23.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:23 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi019 to 4273M 2024-04-25T12:03:23.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:23 smithi019 ceph-mon[29213]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T12:03:23.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:03:23.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:23.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:03:23.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:23.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:23 smithi169 ceph-mon[32710]: Detected new or changed devices on smithi019 2024-04-25T12:03:23.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:23 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi019 to 4273M 2024-04-25T12:03:23.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:23 smithi169 ceph-mon[32710]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T12:03:23.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:03:23.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:23.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:03:23.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:24.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.111+0000 7feb9f106700 1 -- 172.21.15.150:0/8511381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb98103540 msgr2=0x7feb98109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:03:24.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.111+0000 7feb9f106700 1 --2- 172.21.15.150:0/8511381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb98103540 0x7feb98109df0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb9000b0d0 tx=0x7feb9000b3e0 comp rx=0 tx=0).stop 2024-04-25T12:03:24.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.111+0000 7feb9f106700 1 -- 172.21.15.150:0/8511381 shutdown_connections 2024-04-25T12:03:24.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.111+0000 7feb9f106700 1 --2- 172.21.15.150:0/8511381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb981019a0 0x7feb98101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:24.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.111+0000 7feb9f106700 1 --2- 172.21.15.150:0/8511381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb98103540 0x7feb98109df0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:24.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.111+0000 7feb9f106700 1 --2- 172.21.15.150:0/8511381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb98102ba0 0x7feb98103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:24.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.111+0000 7feb9f106700 1 -- 172.21.15.150:0/8511381 >> 172.21.15.150:0/8511381 conn(0x7feb980fcf40 msgr2=0x7feb980ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:03:24.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.111+0000 7feb9f106700 1 -- 172.21.15.150:0/8511381 shutdown_connections 2024-04-25T12:03:24.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.111+0000 7feb9f106700 1 -- 172.21.15.150:0/8511381 wait complete. 2024-04-25T12:03:24.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9f106700 1 Processor -- start 2024-04-25T12:03:24.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9f106700 1 -- start start 2024-04-25T12:03:24.124 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9f106700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb981019a0 0x7feb98198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:03:24.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9f106700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb98102ba0 0x7feb98199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:03:24.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9f106700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb98103540 0x7feb981a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:03:24.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9f106700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feb9810d560 con 0x7feb98103540 2024-04-25T12:03:24.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9f106700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feb9810d3e0 con 0x7feb981019a0 2024-04-25T12:03:24.125 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9f106700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feb9810d6e0 con 0x7feb98102ba0 2024-04-25T12:03:24.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9cea2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb981019a0 0x7feb98198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:03:24.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9cea2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb981019a0 0x7feb98198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37262/0 (socket says 172.21.15.150:37262) 2024-04-25T12:03:24.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9cea2700 1 -- 172.21.15.150:0/2061499481 learned_addr learned my addr 172.21.15.150:0/2061499481 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-25T12:03:24.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9d6a3700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb98103540 0x7feb981a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:03:24.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb97fff700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb98102ba0 0x7feb98199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:03:24.126 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9cea2700 1 -- 172.21.15.150:0/2061499481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb98102ba0 msgr2=0x7feb98199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:03:24.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9cea2700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb98102ba0 0x7feb98199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:24.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9cea2700 1 -- 172.21.15.150:0/2061499481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb98103540 msgr2=0x7feb981a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:03:24.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9cea2700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb98103540 0x7feb981a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:24.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9cea2700 1 -- 172.21.15.150:0/2061499481 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb981a0870 con 0x7feb981019a0 2024-04-25T12:03:24.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb97fff700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb98102ba0 0x7feb98199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:03:24.127 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.112+0000 7feb9cea2700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb981019a0 0x7feb98198c20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00d980 tx=0x7feb8c00dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:03:24.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.113+0000 7feb95ffb700 1 -- 172.21.15.150:0/2061499481 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c017030 con 0x7feb981019a0 2024-04-25T12:03:24.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.113+0000 7feb95ffb700 1 -- 172.21.15.150:0/2061499481 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7feb8c0044f0 con 0x7feb981019a0 2024-04-25T12:03:24.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.113+0000 7feb9f106700 1 -- 172.21.15.150:0/2061499481 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb981a0b00 con 0x7feb981019a0 2024-04-25T12:03:24.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.113+0000 7feb95ffb700 1 -- 172.21.15.150:0/2061499481 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c003b40 con 0x7feb981019a0 2024-04-25T12:03:24.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.113+0000 7feb9f106700 1 -- 172.21.15.150:0/2061499481 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb981a1040 con 0x7feb981019a0 2024-04-25T12:03:24.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.114+0000 7feb9f106700 1 -- 172.21.15.150:0/2061499481 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb98069730 con 0x7feb981019a0 2024-04-25T12:03:24.128 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.115+0000 7feb95ffb700 1 -- 172.21.15.150:0/2061499481 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feb8c01a070 con 0x7feb981019a0 2024-04-25T12:03:24.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.115+0000 7feb95ffb700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7feb880a3c40 0x7feb880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:03:24.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.116+0000 7feb95ffb700 1 -- 172.21.15.150:0/2061499481 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7feb8c0cf9a0 con 0x7feb981019a0 2024-04-25T12:03:24.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.116+0000 7feb97fff700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7feb880a3c40 0x7feb880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:03:24.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.116+0000 7feb97fff700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7feb880a3c40 0x7feb880a6100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feb84005fd0 tx=0x7feb84005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:03:24.129 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.119+0000 7feb95ffb700 1 -- 172.21.15.150:0/2061499481 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb8c09cb80 con 0x7feb981019a0 2024-04-25T12:03:24.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:24.323+0000 7feb9f106700 1 -- 172.21.15.150:0/2061499481 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7feb9806b880 con 0x7feb880a3c40 2024-04-25T12:03:24.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:24 smithi150 ceph-mon[32250]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:24.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:24 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:03:24.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:24 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:03:24.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:24 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:24.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:24 smithi019 ceph-mon[29213]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:24.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:24 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:03:24.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:24 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:03:24.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:24 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:24.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:24 smithi169 ceph-mon[32710]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:24.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:24 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:03:24.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:24 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:03:24.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:24 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:25.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:25 smithi019 ceph-mon[29213]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:03:25.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:25 smithi169 ceph-mon[32710]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:03:25.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:25 smithi150 ceph-mon[32250]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:03:26.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:26 smithi150 ceph-mon[32250]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:26.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:26 smithi019 ceph-mon[29213]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:26.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:26 smithi169 ceph-mon[32710]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:27.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:27 smithi019 ceph-mon[29213]: from='client.? 172.21.15.150:0/2363971054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e97514b7-bafc-4e07-8afb-3d33ab586ef9"}]: dispatch 2024-04-25T12:03:27.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:27 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e97514b7-bafc-4e07-8afb-3d33ab586ef9"}]: dispatch 2024-04-25T12:03:27.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:27 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e97514b7-bafc-4e07-8afb-3d33ab586ef9"}]': finished 2024-04-25T12:03:27.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:27 smithi019 ceph-mon[29213]: osdmap e69: 11 total, 10 up, 11 in 2024-04-25T12:03:27.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:27 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:27.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:27 smithi169 ceph-mon[32710]: from='client.? 172.21.15.150:0/2363971054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e97514b7-bafc-4e07-8afb-3d33ab586ef9"}]: dispatch 2024-04-25T12:03:27.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:27 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e97514b7-bafc-4e07-8afb-3d33ab586ef9"}]: dispatch 2024-04-25T12:03:27.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:27 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e97514b7-bafc-4e07-8afb-3d33ab586ef9"}]': finished 2024-04-25T12:03:27.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:27 smithi169 ceph-mon[32710]: osdmap e69: 11 total, 10 up, 11 in 2024-04-25T12:03:27.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:27 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:27.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:27 smithi150 ceph-mon[32250]: from='client.? 172.21.15.150:0/2363971054' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e97514b7-bafc-4e07-8afb-3d33ab586ef9"}]: dispatch 2024-04-25T12:03:27.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:27 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e97514b7-bafc-4e07-8afb-3d33ab586ef9"}]: dispatch 2024-04-25T12:03:27.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:27 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e97514b7-bafc-4e07-8afb-3d33ab586ef9"}]': finished 2024-04-25T12:03:27.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:27 smithi150 ceph-mon[32250]: osdmap e69: 11 total, 10 up, 11 in 2024-04-25T12:03:27.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:27 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:28.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:28 smithi019 ceph-mon[29213]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:28.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:28 smithi019 ceph-mon[29213]: from='client.? 172.21.15.150:0/1171215103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:03:28.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:28 smithi169 ceph-mon[32710]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:28.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:28 smithi169 ceph-mon[32710]: from='client.? 172.21.15.150:0/1171215103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:03:28.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:28 smithi150 ceph-mon[32250]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:28.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:28 smithi150 ceph-mon[32250]: from='client.? 172.21.15.150:0/1171215103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:03:30.731 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:30 smithi150 ceph-mon[32250]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:30.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:30 smithi019 ceph-mon[29213]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:30.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:30 smithi169 ceph-mon[32710]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:32.658 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:32 smithi150 ceph-mon[32250]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:32.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:32 smithi019 ceph-mon[29213]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:32.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:32 smithi169 ceph-mon[32710]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:34.607 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:34 smithi150 ceph-mon[32250]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:34.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:34 smithi019 ceph-mon[29213]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:34.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:34 smithi169 ceph-mon[32710]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:35.535 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:35 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T12:03:35.535 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:35 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:35.535 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:35 smithi150 ceph-mon[32250]: Deploying daemon osd.10 on smithi150 2024-04-25T12:03:35.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T12:03:35.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:35.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:35 smithi019 ceph-mon[29213]: Deploying daemon osd.10 on smithi150 2024-04-25T12:03:35.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:35 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T12:03:35.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:35 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:35.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:35 smithi169 ceph-mon[32710]: Deploying daemon osd.10 on smithi150 2024-04-25T12:03:36.566 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:36 smithi150 ceph-mon[32250]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:36.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:36 smithi019 ceph-mon[29213]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:36.806 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:36 smithi169 ceph-mon[32710]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:38.748 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:38 smithi150 ceph-mon[32250]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:38.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:38 smithi019 ceph-mon[29213]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:38.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:38 smithi169 ceph-mon[32710]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:40.545 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:40 smithi150 ceph-mon[32250]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:40.796 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:40 smithi019 ceph-mon[29213]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:40.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:40 smithi169 ceph-mon[32710]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:41.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:41 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:41.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:41 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:41.958 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:41 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:03:42.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:41 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:42.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:41 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:42.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:41 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:03:42.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:41 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:42.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:41 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:42.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:41 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:03:42.837 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:42 smithi150 ceph-mon[32250]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:43.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:42 smithi019 ceph-mon[29213]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:43.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:42 smithi169 ceph-mon[32710]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:44.644 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:44.647+0000 7feb95ffb700 1 -- 172.21.15.150:0/2061499481 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7feb9806b880 con 0x7feb880a3c40 2024-04-25T12:03:44.645 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 10 on host 'smithi150' 2024-04-25T12:03:44.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:44.650+0000 7feb9f106700 1 -- 172.21.15.150:0/2061499481 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7feb880a3c40 msgr2=0x7feb880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:03:44.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:44.650+0000 7feb9f106700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7feb880a3c40 0x7feb880a6100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feb84005fd0 tx=0x7feb84005ee0 comp rx=0 tx=0).stop 2024-04-25T12:03:44.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:44.650+0000 7feb9f106700 1 -- 172.21.15.150:0/2061499481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb981019a0 msgr2=0x7feb98198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:03:44.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:44.650+0000 7feb9f106700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb981019a0 0x7feb98198c20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00d980 tx=0x7feb8c00dd40 comp rx=0 tx=0).stop 2024-04-25T12:03:44.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:44.650+0000 7feb9f106700 1 -- 172.21.15.150:0/2061499481 shutdown_connections 2024-04-25T12:03:44.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:44.650+0000 7feb9f106700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7feb880a3c40 0x7feb880a6100 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:44.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:44.650+0000 7feb9f106700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb98103540 0x7feb981a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:44.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:44.650+0000 7feb9f106700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb981019a0 0x7feb98198c20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:44.647 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:44.650+0000 7feb9f106700 1 --2- 172.21.15.150:0/2061499481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feb98102ba0 0x7feb98199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:44.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:44.650+0000 7feb9f106700 1 -- 172.21.15.150:0/2061499481 >> 172.21.15.150:0/2061499481 conn(0x7feb980fcf40 msgr2=0x7feb9806b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:03:44.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:44.650+0000 7feb9f106700 1 -- 172.21.15.150:0/2061499481 shutdown_connections 2024-04-25T12:03:44.648 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-25T12:03:44.650+0000 7feb9f106700 1 -- 172.21.15.150:0/2061499481 wait complete. 2024-04-25T12:03:44.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:44 smithi150 ceph-mon[32250]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:44.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:44 smithi150 ceph-mon[32250]: from='osd.10 [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T12:03:44.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:44 smithi150 ceph-mon[32250]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T12:03:44.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:44 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:44.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:44 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:45.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:44 smithi019 ceph-mon[29213]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:45.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:44 smithi019 ceph-mon[29213]: from='osd.10 [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T12:03:45.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:44 smithi019 ceph-mon[29213]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T12:03:45.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:44 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:45.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:44 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:45.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:44 smithi169 ceph-mon[32710]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:45.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:44 smithi169 ceph-mon[32710]: from='osd.10 [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T12:03:45.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:44 smithi169 ceph-mon[32710]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T12:03:45.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:44 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:45.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:44 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:45.410 DEBUG:teuthology.orchestra.run.smithi150:osd.10> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T12:03:45.412 INFO:tasks.cephadm:Deploying osd.11 on smithi169 with /dev/vg_nvme/lv_1... 2024-04-25T12:03:45.412 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T12:03:45.669 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:03:46.041 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:45 smithi169 ceph-mon[32710]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T12:03:46.042 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:45 smithi169 ceph-mon[32710]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T12:03:46.042 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:46.042 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:45 smithi169 ceph-mon[32710]: from='osd.10 [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:03:46.042 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:45 smithi169 ceph-mon[32710]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:03:46.042 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:46.042 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:45 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:46.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:45 smithi019 ceph-mon[29213]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T12:03:46.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:45 smithi019 ceph-mon[29213]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T12:03:46.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:46.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:45 smithi019 ceph-mon[29213]: from='osd.10 [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:03:46.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:45 smithi019 ceph-mon[29213]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:03:46.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:46.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:45 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:46.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:45 smithi150 ceph-mon[32250]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T12:03:46.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:45 smithi150 ceph-mon[32250]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T12:03:46.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:45 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:46.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:45 smithi150 ceph-mon[32250]: from='osd.10 [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:03:46.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:45 smithi150 ceph-mon[32250]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-25T12:03:46.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:45 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:46.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:45 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:46.590 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:03:46 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:03:46.165+0000 7f7c4a1a2700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T12:03:46.591 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:03:46 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:03:46.179+0000 7f7c4178c700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:03:46.880 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:46 smithi169 ceph-mon[32710]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:46.880 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:46 smithi169 ceph-mon[32710]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T12:03:46.880 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:46 smithi169 ceph-mon[32710]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T12:03:46.880 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:46 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:46.881 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:46 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:46.881 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:46 smithi169 ceph-mon[32710]: from='osd.10 ' entity='osd.10' 2024-04-25T12:03:46.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:46 smithi150 ceph-mon[32250]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:46.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:46 smithi150 ceph-mon[32250]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T12:03:46.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:46 smithi150 ceph-mon[32250]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T12:03:46.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:46 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:46.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:46 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:46.909 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:46 smithi150 ceph-mon[32250]: from='osd.10 ' entity='osd.10' 2024-04-25T12:03:47.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:46 smithi019 ceph-mon[29213]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-25T12:03:47.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:46 smithi019 ceph-mon[29213]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-25T12:03:47.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:46 smithi019 ceph-mon[29213]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T12:03:47.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:46 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:47.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:46 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:47.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:46 smithi019 ceph-mon[29213]: from='osd.10 ' entity='osd.10' 2024-04-25T12:03:47.762 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T12:03:47.762 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T12:03:47.762 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-04-25T12:03:47.763 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-04-25T12:03:47.763 INFO:teuthology.orchestra.run.smithi169.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011853 s, 885 MB/s 2024-04-25T12:03:47.763 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-04-25T12:03:48.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:47 smithi019 ceph-mon[29213]: purged_snaps scrub starts 2024-04-25T12:03:48.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:47 smithi019 ceph-mon[29213]: purged_snaps scrub ok 2024-04-25T12:03:48.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:47 smithi019 ceph-mon[29213]: osd.10 [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024] boot 2024-04-25T12:03:48.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:47 smithi019 ceph-mon[29213]: osdmap e72: 11 total, 11 up, 11 in 2024-04-25T12:03:48.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:48.095 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:47 smithi169 ceph-mon[32710]: purged_snaps scrub starts 2024-04-25T12:03:48.095 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:47 smithi169 ceph-mon[32710]: purged_snaps scrub ok 2024-04-25T12:03:48.095 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:47 smithi169 ceph-mon[32710]: osd.10 [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024] boot 2024-04-25T12:03:48.096 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:47 smithi169 ceph-mon[32710]: osdmap e72: 11 total, 11 up, 11 in 2024-04-25T12:03:48.096 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:48.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:47 smithi150 ceph-mon[32250]: purged_snaps scrub starts 2024-04-25T12:03:48.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:47 smithi150 ceph-mon[32250]: purged_snaps scrub ok 2024-04-25T12:03:48.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:47 smithi150 ceph-mon[32250]: osd.10 [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024] boot 2024-04-25T12:03:48.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:47 smithi150 ceph-mon[32250]: osdmap e72: 11 total, 11 up, 11 in 2024-04-25T12:03:48.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:03:48.492 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi169:vg_nvme/lv_1 2024-04-25T12:03:48.737 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:03:48.993 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:48 smithi169 ceph-mon[32710]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:03:48.993 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:48 smithi169 ceph-mon[32710]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T12:03:49.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:48 smithi019 ceph-mon[29213]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:03:49.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:48 smithi019 ceph-mon[29213]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T12:03:49.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:48 smithi150 ceph-mon[32250]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:03:49.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:48 smithi150 ceph-mon[32250]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T12:03:50.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: Detected new or changed devices on smithi150 2024-04-25T12:03:50.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T12:03:50.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:50.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:50.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:50.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:50.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi150 to 4272M 2024-04-25T12:03:50.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:50.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:03:50.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:03:50.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:50.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:03:50.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.087 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: Detected new or changed devices on smithi150 2024-04-25T12:03:50.087 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T12:03:50.087 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.087 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.087 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:50.087 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:50.087 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:50.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:50.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi150 to 4272M 2024-04-25T12:03:50.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:50.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:03:50.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:03:50.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:50.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:03:50.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.137 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: Detected new or changed devices on smithi150 2024-04-25T12:03:50.137 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T12:03:50.137 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.137 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.137 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:50.137 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:50.137 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:50.137 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:03:50.137 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi150 to 4272M 2024-04-25T12:03:50.137 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.137 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:50.138 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:03:50.138 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.138 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:03:50.138 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:50.138 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:03:50.138 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:03:50.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.508+0000 7fa73268f700 1 -- 172.21.15.169:0/1323785931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa72c103550 msgr2=0x7fa72c109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:03:50.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.508+0000 7fa73268f700 1 --2- 172.21.15.169:0/1323785931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa72c103550 0x7fa72c109e00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa72400b0a0 tx=0x7fa72400b3b0 comp rx=0 tx=0).stop 2024-04-25T12:03:50.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.508+0000 7fa73268f700 1 -- 172.21.15.169:0/1323785931 shutdown_connections 2024-04-25T12:03:50.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.508+0000 7fa73268f700 1 --2- 172.21.15.169:0/1323785931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa72c103550 0x7fa72c109e00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:50.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.508+0000 7fa73268f700 1 --2- 172.21.15.169:0/1323785931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c1019b0 0x7fa72c101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:50.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.508+0000 7fa73268f700 1 --2- 172.21.15.169:0/1323785931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa72c102bb0 0x7fa72c103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.508+0000 7fa73268f700 1 -- 172.21.15.169:0/1323785931 >> 172.21.15.169:0/1323785931 conn(0x7fa72c0fcf70 msgr2=0x7fa72c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:03:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.508+0000 7fa73268f700 1 -- 172.21.15.169:0/1323785931 shutdown_connections 2024-04-25T12:03:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.508+0000 7fa73268f700 1 -- 172.21.15.169:0/1323785931 wait complete. 2024-04-25T12:03:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.509+0000 7fa73268f700 1 Processor -- start 2024-04-25T12:03:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.509+0000 7fa73268f700 1 -- start start 2024-04-25T12:03:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.509+0000 7fa73268f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c1019b0 0x7fa72c1059b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:03:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.509+0000 7fa73268f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa72c102bb0 0x7fa72c104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:03:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.509+0000 7fa73268f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa72c103550 0x7fa72c104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:03:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.509+0000 7fa73268f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c10d5c0 con 0x7fa72c102bb0 2024-04-25T12:03:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.509+0000 7fa73268f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c10d440 con 0x7fa72c1019b0 2024-04-25T12:03:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.509+0000 7fa73268f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c10d740 con 0x7fa72c103550 2024-04-25T12:03:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa730c2c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa72c103550 0x7fa72c104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:03:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa730c2c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa72c103550 0x7fa72c104540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:42712/0 (socket says 172.21.15.169:42712) 2024-04-25T12:03:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa730c2c700 1 -- 172.21.15.169:0/1382316034 learned_addr learned my addr 172.21.15.169:0/1382316034 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T12:03:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa72bfff700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c1019b0 0x7fa72c1059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:03:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa72b7fe700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa72c102bb0 0x7fa72c104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:03:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa730c2c700 1 -- 172.21.15.169:0/1382316034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c1019b0 msgr2=0x7fa72c1059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:03:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa730c2c700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c1019b0 0x7fa72c1059b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa730c2c700 1 -- 172.21.15.169:0/1382316034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa72c102bb0 msgr2=0x7fa72c104000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:03:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa730c2c700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa72c102bb0 0x7fa72c104000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:03:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa730c2c700 1 -- 172.21.15.169:0/1382316034 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72c104df0 con 0x7fa72c103550 2024-04-25T12:03:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa72b7fe700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa72c102bb0 0x7fa72c104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:03:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa730c2c700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa72c103550 0x7fa72c104540 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa72400b070 tx=0x7fa724007d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:03:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa7297fa700 1 -- 172.21.15.169:0/1382316034 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa724004e60 con 0x7fa72c103550 2024-04-25T12:03:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa7297fa700 1 -- 172.21.15.169:0/1382316034 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa724022070 con 0x7fa72c103550 2024-04-25T12:03:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa73268f700 1 -- 172.21.15.169:0/1382316034 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72c1a2d70 con 0x7fa72c103550 2024-04-25T12:03:50.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa7297fa700 1 -- 172.21.15.169:0/1382316034 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72401b9a0 con 0x7fa72c103550 2024-04-25T12:03:50.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.510+0000 7fa73268f700 1 -- 172.21.15.169:0/1382316034 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72c1a31c0 con 0x7fa72c103550 2024-04-25T12:03:50.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.511+0000 7fa73268f700 1 -- 172.21.15.169:0/1382316034 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa72c1a3390 con 0x7fa72c103550 2024-04-25T12:03:50.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.512+0000 7fa7297fa700 1 -- 172.21.15.169:0/1382316034 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa72401dbb0 con 0x7fa72c103550 2024-04-25T12:03:50.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.513+0000 7fa7297fa700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa7140a3c40 0x7fa7140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:03:50.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.513+0000 7fa7297fa700 1 -- 172.21.15.169:0/1382316034 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7fa72400f020 con 0x7fa72c103550 2024-04-25T12:03:50.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.513+0000 7fa72bfff700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa7140a3c40 0x7fa7140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:03:50.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.514+0000 7fa72bfff700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa7140a3c40 0x7fa7140a6100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa72c102a10 tx=0x7fa71c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:03:50.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.517+0000 7fa7297fa700 1 -- 172.21.15.169:0/1382316034 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa72409def0 con 0x7fa72c103550 2024-04-25T12:03:50.725 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:03:50.727+0000 7fa73268f700 1 -- 172.21.15.169:0/1382316034 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa72c06b8a0 con 0x7fa7140a3c40 2024-04-25T12:03:51.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:50 smithi019 ceph-mon[29213]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:03:51.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:50 smithi019 ceph-mon[29213]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T12:03:51.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:50 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:03:51.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:50 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:03:51.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:50 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:51.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:50 smithi169 ceph-mon[32710]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:03:51.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:50 smithi169 ceph-mon[32710]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T12:03:51.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:50 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:03:51.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:50 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:03:51.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:50 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:51.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:50 smithi150 ceph-mon[32250]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:03:51.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:50 smithi150 ceph-mon[32250]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T12:03:51.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:50 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T12:03:51.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:50 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T12:03:51.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:50 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:03:51.904 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:51 smithi169 ceph-mon[32710]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:03:51.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:51 smithi169 ceph-mon[32710]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T12:03:52.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:51 smithi019 ceph-mon[29213]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:03:52.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:51 smithi019 ceph-mon[29213]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T12:03:52.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:51 smithi150 ceph-mon[32250]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:03:52.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:51 smithi150 ceph-mon[32250]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T12:03:52.987 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:52 smithi169 ceph-mon[32710]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-25T12:03:53.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:52 smithi019 ceph-mon[29213]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-25T12:03:53.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:52 smithi150 ceph-mon[32250]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-25T12:03:54.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:53 smithi019 ceph-mon[29213]: from='client.? 172.21.15.169:0/3984016789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98d4c806-cf93-41d4-9b62-9febe86e9372"}]: dispatch 2024-04-25T12:03:54.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:53 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98d4c806-cf93-41d4-9b62-9febe86e9372"}]: dispatch 2024-04-25T12:03:54.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:53 smithi019 ceph-mon[29213]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98d4c806-cf93-41d4-9b62-9febe86e9372"}]': finished 2024-04-25T12:03:54.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:53 smithi019 ceph-mon[29213]: osdmap e77: 12 total, 11 up, 12 in 2024-04-25T12:03:54.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:53 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:03:54.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:53 smithi169 ceph-mon[32710]: from='client.? 172.21.15.169:0/3984016789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98d4c806-cf93-41d4-9b62-9febe86e9372"}]: dispatch 2024-04-25T12:03:54.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:53 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98d4c806-cf93-41d4-9b62-9febe86e9372"}]: dispatch 2024-04-25T12:03:54.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:53 smithi169 ceph-mon[32710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98d4c806-cf93-41d4-9b62-9febe86e9372"}]': finished 2024-04-25T12:03:54.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:53 smithi169 ceph-mon[32710]: osdmap e77: 12 total, 11 up, 12 in 2024-04-25T12:03:54.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:53 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:03:54.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:53 smithi150 ceph-mon[32250]: from='client.? 172.21.15.169:0/3984016789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98d4c806-cf93-41d4-9b62-9febe86e9372"}]: dispatch 2024-04-25T12:03:54.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:53 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98d4c806-cf93-41d4-9b62-9febe86e9372"}]: dispatch 2024-04-25T12:03:54.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:53 smithi150 ceph-mon[32250]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98d4c806-cf93-41d4-9b62-9febe86e9372"}]': finished 2024-04-25T12:03:54.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:53 smithi150 ceph-mon[32250]: osdmap e77: 12 total, 11 up, 12 in 2024-04-25T12:03:54.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:53 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:03:55.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:54 smithi019 ceph-mon[29213]: pgmap v245: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-25T12:03:55.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:54 smithi019 ceph-mon[29213]: from='client.? 172.21.15.169:0/3317974061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:03:55.047 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:03:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:03:54.837+0000 7efe5b4b4700 -1 calc_pg_upmaps abort due to max <= 0 2024-04-25T12:03:55.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:54 smithi169 ceph-mon[32710]: pgmap v245: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-25T12:03:55.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:54 smithi169 ceph-mon[32710]: from='client.? 172.21.15.169:0/3317974061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:03:55.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:54 smithi150 ceph-mon[32250]: pgmap v245: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-04-25T12:03:55.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:54 smithi150 ceph-mon[32250]: from='client.? 172.21.15.169:0/3317974061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T12:03:56.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:55 smithi019 ceph-mon[29213]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T12:03:56.154 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:55 smithi169 ceph-mon[32710]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T12:03:56.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:55 smithi150 ceph-mon[32250]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T12:03:58.102 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:57 smithi169 ceph-mon[32710]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:03:58.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:57 smithi019 ceph-mon[29213]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:03:58.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:57 smithi150 ceph-mon[32250]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:00.077 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:03:59 smithi169 ceph-mon[32710]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:00.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:03:59 smithi019 ceph-mon[29213]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:00.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:03:59 smithi150 ceph-mon[32250]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:02.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:01 smithi169 ceph-mon[32710]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:02.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T12:04:02.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:02.197 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:01 smithi169 ceph-mon[32710]: Deploying daemon osd.11 on smithi169 2024-04-25T12:04:02.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:01 smithi019 ceph-mon[29213]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:02.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T12:04:02.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:02.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:01 smithi019 ceph-mon[29213]: Deploying daemon osd.11 on smithi169 2024-04-25T12:04:02.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:01 smithi150 ceph-mon[32250]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:02.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T12:04:02.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:02.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:01 smithi150 ceph-mon[32250]: Deploying daemon osd.11 on smithi169 2024-04-25T12:04:04.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:03 smithi019 ceph-mon[29213]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:04.404 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:03 smithi169 ceph-mon[32710]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:04.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:03 smithi150 ceph-mon[32250]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:06.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:05 smithi019 ceph-mon[29213]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:06.397 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:05 smithi169 ceph-mon[32710]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:06.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:05 smithi150 ceph-mon[32250]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:08.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:08 smithi019 ceph-mon[29213]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:08.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:08.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:08.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:08 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:04:08.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:08 smithi169 ceph-mon[32710]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:08.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:08.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:08.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:08 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:04:08.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:08 smithi150 ceph-mon[32250]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:08.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:08.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:08.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:08 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:04:10.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:10 smithi019 ceph-mon[29213]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:10.864 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:10 smithi169 ceph-mon[32710]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:10.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:10 smithi150 ceph-mon[32250]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:10.944 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:10.947+0000 7fa7297fa700 1 -- 172.21.15.169:0/1382316034 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa72c06b8a0 con 0x7fa7140a3c40 2024-04-25T12:04:10.945 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 11 on host 'smithi169' 2024-04-25T12:04:10.947 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:10.949+0000 7fa73268f700 1 -- 172.21.15.169:0/1382316034 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa7140a3c40 msgr2=0x7fa7140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:10.947 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:10.949+0000 7fa73268f700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa7140a3c40 0x7fa7140a6100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa72c102a10 tx=0x7fa71c00a380 comp rx=0 tx=0).stop 2024-04-25T12:04:10.947 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:10.949+0000 7fa73268f700 1 -- 172.21.15.169:0/1382316034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa72c103550 msgr2=0x7fa72c104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:10.947 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:10.949+0000 7fa73268f700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa72c103550 0x7fa72c104540 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa72400b070 tx=0x7fa724007d10 comp rx=0 tx=0).stop 2024-04-25T12:04:10.947 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:10.950+0000 7fa73268f700 1 -- 172.21.15.169:0/1382316034 shutdown_connections 2024-04-25T12:04:10.947 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:10.950+0000 7fa73268f700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa7140a3c40 0x7fa7140a6100 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:10.947 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:10.950+0000 7fa73268f700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa72c103550 0x7fa72c104540 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:10.948 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:10.950+0000 7fa73268f700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c1019b0 0x7fa72c1059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:10.948 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:10.950+0000 7fa73268f700 1 --2- 172.21.15.169:0/1382316034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa72c102bb0 0x7fa72c104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:10.948 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:10.950+0000 7fa73268f700 1 -- 172.21.15.169:0/1382316034 >> 172.21.15.169:0/1382316034 conn(0x7fa72c0fcf70 msgr2=0x7fa72c06b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:10.948 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:10.950+0000 7fa73268f700 1 -- 172.21.15.169:0/1382316034 shutdown_connections 2024-04-25T12:04:10.948 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:10.950+0000 7fa73268f700 1 -- 172.21.15.169:0/1382316034 wait complete. 2024-04-25T12:04:11.563 DEBUG:teuthology.orchestra.run.smithi169:osd.11> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T12:04:11.565 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T12:04:11.566 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T12:04:11.838 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:04:12.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:11 smithi169 ceph-mon[32710]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:12.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:11 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:12.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:11 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:12.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:11 smithi169 ceph-mon[32710]: from='osd.11 [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T12:04:12.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:11 smithi169 ceph-mon[32710]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T12:04:12.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:11 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:12.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:11 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:12.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:11 smithi150 ceph-mon[32250]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:12.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:11 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:12.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:11 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:12.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:11 smithi150 ceph-mon[32250]: from='osd.11 [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T12:04:12.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:11 smithi150 ceph-mon[32250]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T12:04:12.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:11 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:12.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:11 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:12.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:11 smithi019 ceph-mon[29213]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:12.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:11 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:12.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:11 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:12.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:11 smithi019 ceph-mon[29213]: from='osd.11 [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T12:04:12.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:11 smithi019 ceph-mon[29213]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T12:04:12.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:11 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:12.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:11 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:13.059 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:12 smithi169 ceph-mon[32710]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T12:04:13.059 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:12 smithi169 ceph-mon[32710]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T12:04:13.059 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:12 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:04:13.059 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:12 smithi169 ceph-mon[32710]: from='osd.11 [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:04:13.059 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:12 smithi169 ceph-mon[32710]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:04:13.059 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:04:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:04:12.916+0000 7fab10922700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T12:04:13.059 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:04:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:04:12.928+0000 7fab0c715700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T12:04:13.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:12 smithi150 ceph-mon[32250]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T12:04:13.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:12 smithi150 ceph-mon[32250]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T12:04:13.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:12 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:04:13.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:12 smithi150 ceph-mon[32250]: from='osd.11 [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:04:13.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:12 smithi150 ceph-mon[32250]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:04:13.208 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:12 smithi019 ceph-mon[29213]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T12:04:13.208 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:12 smithi019 ceph-mon[29213]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T12:04:13.208 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:12 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:04:13.208 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:12 smithi019 ceph-mon[29213]: from='osd.11 [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:04:13.208 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:12 smithi019 ceph-mon[29213]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-04-25T12:04:13.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.771+0000 7fd34e5a7700 1 -- 172.21.15.19:0/915929637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd348101320 msgr2=0x7fd348101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:13.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.771+0000 7fd34e5a7700 1 --2- 172.21.15.19:0/915929637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd348101320 0x7fd348101740 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd338009990 tx=0x7fd338009ca0 comp rx=0 tx=0).stop 2024-04-25T12:04:13.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.771+0000 7fd34e5a7700 1 -- 172.21.15.19:0/915929637 shutdown_connections 2024-04-25T12:04:13.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.771+0000 7fd34e5a7700 1 --2- 172.21.15.19:0/915929637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd348101320 0x7fd348101740 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:13.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.771+0000 7fd34e5a7700 1 --2- 172.21.15.19:0/915929637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd348102640 0x7fd348102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:13.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.771+0000 7fd34e5a7700 1 --2- 172.21.15.19:0/915929637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd348101c80 0x7fd348102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:13.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.771+0000 7fd34e5a7700 1 -- 172.21.15.19:0/915929637 >> 172.21.15.19:0/915929637 conn(0x7fd3480fcf40 msgr2=0x7fd3480ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:13.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.771+0000 7fd34e5a7700 1 -- 172.21.15.19:0/915929637 shutdown_connections 2024-04-25T12:04:13.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.771+0000 7fd34e5a7700 1 -- 172.21.15.19:0/915929637 wait complete. 2024-04-25T12:04:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.771+0000 7fd34e5a7700 1 Processor -- start 2024-04-25T12:04:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.771+0000 7fd34e5a7700 1 -- start start 2024-04-25T12:04:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd34e5a7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd348101320 0x7fd34819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd34e5a7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd348101c80 0x7fd34819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd34e5a7700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd348102640 0x7fd3481a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd34e5a7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3481108b0 con 0x7fd348101c80 2024-04-25T12:04:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd34e5a7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd348110730 con 0x7fd348101320 2024-04-25T12:04:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd34e5a7700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd348110a30 con 0x7fd348102640 2024-04-25T12:04:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd3477fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd348101c80 0x7fd34819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd3477fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd348101c80 0x7fd34819b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59224/0 (socket says 172.21.15.19:59224) 2024-04-25T12:04:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd3477fe700 1 -- 172.21.15.19:0/3688860454 learned_addr learned my addr 172.21.15.19:0/3688860454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd34cb44700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd348102640 0x7fd3481a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:13.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd347fff700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd348101320 0x7fd34819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:13.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd3477fe700 1 -- 172.21.15.19:0/3688860454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd348102640 msgr2=0x7fd3481a24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:13.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd3477fe700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd348102640 0x7fd3481a24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:13.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd3477fe700 1 -- 172.21.15.19:0/3688860454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd348101320 msgr2=0x7fd34819aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:13.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd3477fe700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd348101320 0x7fd34819aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:13.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd3477fe700 1 -- 172.21.15.19:0/3688860454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3481a2b50 con 0x7fd348101c80 2024-04-25T12:04:13.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd347fff700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd348101320 0x7fd34819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:04:13.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd3477fe700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd348101c80 0x7fd34819b430 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00b560 tx=0x7fd33c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:13.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd3457fa700 1 -- 172.21.15.19:0/3688860454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c015020 con 0x7fd348101c80 2024-04-25T12:04:13.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd34e5a7700 1 -- 172.21.15.19:0/3688860454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3481a2e40 con 0x7fd348101c80 2024-04-25T12:04:13.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.772+0000 7fd3457fa700 1 -- 172.21.15.19:0/3688860454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd33c011d00 con 0x7fd348101c80 2024-04-25T12:04:13.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.773+0000 7fd3457fa700 1 -- 172.21.15.19:0/3688860454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c010e60 con 0x7fd348101c80 2024-04-25T12:04:13.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.773+0000 7fd34e5a7700 1 -- 172.21.15.19:0/3688860454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3481a33c0 con 0x7fd348101c80 2024-04-25T12:04:13.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.774+0000 7fd34e5a7700 1 -- 172.21.15.19:0/3688860454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd348069730 con 0x7fd348101c80 2024-04-25T12:04:13.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.775+0000 7fd3457fa700 1 -- 172.21.15.19:0/3688860454 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd33c018020 con 0x7fd348101c80 2024-04-25T12:04:13.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.775+0000 7fd3457fa700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd3300a3c40 0x7fd3300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:13.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.775+0000 7fd347fff700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd3300a3c40 0x7fd3300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:13.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.775+0000 7fd3457fa700 1 -- 172.21.15.19:0/3688860454 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7fd33c0cf570 con 0x7fd348101c80 2024-04-25T12:04:13.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.776+0000 7fd347fff700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd3300a3c40 0x7fd3300a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd338006010 tx=0x7fd338003800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:13.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.780+0000 7fd3457fa700 1 -- 172.21.15.19:0/3688860454 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd33c01f020 con 0x7fd348101c80 2024-04-25T12:04:13.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.992+0000 7fd34e5a7700 1 -- 172.21.15.19:0/3688860454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fd34802e000 con 0x7fd348101c80 2024-04-25T12:04:13.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.992+0000 7fd3457fa700 1 -- 172.21.15.19:0/3688860454 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7fd33c09c510 con 0x7fd348101c80 2024-04-25T12:04:13.991 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:13.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.994+0000 7fd34e5a7700 1 -- 172.21.15.19:0/3688860454 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd3300a3c40 msgr2=0x7fd3300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:13.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.995+0000 7fd34e5a7700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd3300a3c40 0x7fd3300a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd338006010 tx=0x7fd338003800 comp rx=0 tx=0).stop 2024-04-25T12:04:13.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.995+0000 7fd34e5a7700 1 -- 172.21.15.19:0/3688860454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd348101c80 msgr2=0x7fd34819b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:13.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.995+0000 7fd34e5a7700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd348101c80 0x7fd34819b430 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00b560 tx=0x7fd33c00b870 comp rx=0 tx=0).stop 2024-04-25T12:04:13.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.995+0000 7fd34e5a7700 1 -- 172.21.15.19:0/3688860454 shutdown_connections 2024-04-25T12:04:13.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.995+0000 7fd34e5a7700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd3300a3c40 0x7fd3300a6100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:13.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.995+0000 7fd34e5a7700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd348102640 0x7fd3481a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:13.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.995+0000 7fd34e5a7700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd348101320 0x7fd34819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:13.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.995+0000 7fd34e5a7700 1 --2- 172.21.15.19:0/3688860454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd348101c80 0x7fd34819b430 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:13.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.995+0000 7fd34e5a7700 1 -- 172.21.15.19:0/3688860454 >> 172.21.15.19:0/3688860454 conn(0x7fd3480fcf40 msgr2=0x7fd3480fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:13.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.995+0000 7fd34e5a7700 1 -- 172.21.15.19:0/3688860454 shutdown_connections 2024-04-25T12:04:13.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:13.995+0000 7fd34e5a7700 1 -- 172.21.15.19:0/3688860454 wait complete. 2024-04-25T12:04:14.289 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:13 smithi019 ceph-mon[29213]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:14.289 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:13 smithi019 ceph-mon[29213]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T12:04:14.289 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:13 smithi019 ceph-mon[29213]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T12:04:14.289 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:13 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:04:14.289 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:13 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:04:14.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:13 smithi169 ceph-mon[32710]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:14.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:13 smithi169 ceph-mon[32710]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T12:04:14.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:13 smithi169 ceph-mon[32710]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T12:04:14.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:13 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:04:14.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:13 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:04:14.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:13 smithi150 ceph-mon[32250]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:14.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:13 smithi150 ceph-mon[32250]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-04-25T12:04:14.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:13 smithi150 ceph-mon[32250]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T12:04:14.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:13 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:04:14.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:13 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:04:14.671 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1714046653,"num_in_osds":12,"osd_in_since":1714046633,"num_remapped_pgs":0} 2024-04-25T12:04:14.672 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T12:04:14.922 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:04:15.152 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:14 smithi169 ceph-mon[32710]: purged_snaps scrub starts 2024-04-25T12:04:15.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:14 smithi169 ceph-mon[32710]: purged_snaps scrub ok 2024-04-25T12:04:15.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:14 smithi169 ceph-mon[32710]: osd.11 [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529] boot 2024-04-25T12:04:15.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:14 smithi169 ceph-mon[32710]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T12:04:15.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:14 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:04:15.153 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:14 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3688860454' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T12:04:15.202 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:14 smithi019 ceph-mon[29213]: purged_snaps scrub starts 2024-04-25T12:04:15.202 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:14 smithi019 ceph-mon[29213]: purged_snaps scrub ok 2024-04-25T12:04:15.202 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:14 smithi019 ceph-mon[29213]: osd.11 [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529] boot 2024-04-25T12:04:15.202 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:14 smithi019 ceph-mon[29213]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T12:04:15.202 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:14 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:04:15.202 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:14 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3688860454' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T12:04:15.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:14 smithi150 ceph-mon[32250]: purged_snaps scrub starts 2024-04-25T12:04:15.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:14 smithi150 ceph-mon[32250]: purged_snaps scrub ok 2024-04-25T12:04:15.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:14 smithi150 ceph-mon[32250]: osd.11 [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529] boot 2024-04-25T12:04:15.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:14 smithi150 ceph-mon[32250]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T12:04:15.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:14 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:04:15.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:14 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3688860454' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T12:04:16.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:16.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T12:04:16.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: Detected new or changed devices on smithi169 2024-04-25T12:04:16.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:16.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:16.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:04:16.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:04:16.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:04:16.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:04:16.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-25T12:04:16.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:16.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:16.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:04:16.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:16.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:15 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:04:16.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:16.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T12:04:16.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: Detected new or changed devices on smithi169 2024-04-25T12:04:16.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:16.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:16.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:04:16.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:04:16.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:04:16.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:04:16.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-25T12:04:16.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:16.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:16.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:04:16.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:16.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:15 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:04:16.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-25T12:04:16.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T12:04:16.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: Detected new or changed devices on smithi169 2024-04-25T12:04:16.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:16.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:16.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:04:16.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:04:16.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:04:16.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:04:16.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi169 to 4272M 2024-04-25T12:04:16.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:16.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:16.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:04:16.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:16.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:15 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:04:16.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.796+0000 7fd6d0878700 1 -- 172.21.15.19:0/3922512731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6cc108a50 msgr2=0x7fd6cc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:16.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.796+0000 7fd6d0878700 1 --2- 172.21.15.19:0/3922512731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6cc108a50 0x7fd6cc108e30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009990 tx=0x7fd6bc009ca0 comp rx=0 tx=0).stop 2024-04-25T12:04:16.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.796+0000 7fd6d0878700 1 -- 172.21.15.19:0/3922512731 shutdown_connections 2024-04-25T12:04:16.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.796+0000 7fd6d0878700 1 --2- 172.21.15.19:0/3922512731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6cc108a50 0x7fd6cc108e30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:16.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.796+0000 7fd6d0878700 1 --2- 172.21.15.19:0/3922512731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc103450 0x7fd6cc10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:16.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.796+0000 7fd6d0878700 1 --2- 172.21.15.19:0/3922512731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc102a90 0x7fd6cc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:16.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.796+0000 7fd6d0878700 1 -- 172.21.15.19:0/3922512731 >> 172.21.15.19:0/3922512731 conn(0x7fd6cc079ba0 msgr2=0x7fd6cc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:16.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.797+0000 7fd6d0878700 1 -- 172.21.15.19:0/3922512731 shutdown_connections 2024-04-25T12:04:16.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.797+0000 7fd6d0878700 1 -- 172.21.15.19:0/3922512731 wait complete. 2024-04-25T12:04:16.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.797+0000 7fd6d0878700 1 Processor -- start 2024-04-25T12:04:16.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.797+0000 7fd6d0878700 1 -- start start 2024-04-25T12:04:16.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6d0878700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc102a90 0x7fd6cc19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:16.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6d0878700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6cc103450 0x7fd6cc19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:16.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6d0878700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc108a50 0x7fd6cc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:16.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6d0878700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc112d60 con 0x7fd6cc103450 2024-04-25T12:04:16.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6d0878700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc112be0 con 0x7fd6cc108a50 2024-04-25T12:04:16.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6d0878700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc112ee0 con 0x7fd6cc102a90 2024-04-25T12:04:16.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6c9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6cc103450 0x7fd6cc19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:16.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6c9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6cc103450 0x7fd6cc19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59242/0 (socket says 172.21.15.19:59242) 2024-04-25T12:04:16.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6c9d9b700 1 -- 172.21.15.19:0/1930487757 learned_addr learned my addr 172.21.15.19:0/1930487757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:16.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6ca59c700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc102a90 0x7fd6cc19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:16.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6cad9d700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc108a50 0x7fd6cc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:16.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6c9d9b700 1 -- 172.21.15.19:0/1930487757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc102a90 msgr2=0x7fd6cc19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:16.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6c9d9b700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc102a90 0x7fd6cc19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:16.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6c9d9b700 1 -- 172.21.15.19:0/1930487757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc108a50 msgr2=0x7fd6cc19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:16.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6c9d9b700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc108a50 0x7fd6cc19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:16.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6c9d9b700 1 -- 172.21.15.19:0/1930487757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6cc1a0420 con 0x7fd6cc103450 2024-04-25T12:04:16.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6ca59c700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc102a90 0x7fd6cc19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:04:16.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6cad9d700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc108a50 0x7fd6cc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:04:16.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.798+0000 7fd6c9d9b700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6cc103450 0x7fd6cc19b9a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000c970 tx=0x7fd6c000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:16.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.799+0000 7fd6bb7fe700 1 -- 172.21.15.19:0/1930487757 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0007aa0 con 0x7fd6cc103450 2024-04-25T12:04:16.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.799+0000 7fd6d0878700 1 -- 172.21.15.19:0/1930487757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6cc1a0710 con 0x7fd6cc103450 2024-04-25T12:04:16.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.799+0000 7fd6bb7fe700 1 -- 172.21.15.19:0/1930487757 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd6c0004d00 con 0x7fd6cc103450 2024-04-25T12:04:16.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.799+0000 7fd6bb7fe700 1 -- 172.21.15.19:0/1930487757 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0005670 con 0x7fd6cc103450 2024-04-25T12:04:16.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.799+0000 7fd6d0878700 1 -- 172.21.15.19:0/1930487757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6cc1a7f90 con 0x7fd6cc103450 2024-04-25T12:04:16.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.799+0000 7fd6d0878700 1 -- 172.21.15.19:0/1930487757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6cc069730 con 0x7fd6cc103450 2024-04-25T12:04:16.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.801+0000 7fd6bb7fe700 1 -- 172.21.15.19:0/1930487757 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd6c0007cd0 con 0x7fd6cc103450 2024-04-25T12:04:16.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.802+0000 7fd6bb7fe700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd6b40a3c40 0x7fd6b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:16.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.802+0000 7fd6bb7fe700 1 -- 172.21.15.19:0/1930487757 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd6c00d0230 con 0x7fd6cc103450 2024-04-25T12:04:16.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.802+0000 7fd6ca59c700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd6b40a3c40 0x7fd6b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:16.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.803+0000 7fd6ca59c700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd6b40a3c40 0x7fd6b40a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00b5c0 tx=0x7fd6bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:16.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:16.806+0000 7fd6bb7fe700 1 -- 172.21.15.19:0/1930487757 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6c009d0a0 con 0x7fd6cc103450 2024-04-25T12:04:17.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:17.015+0000 7fd6d0878700 1 -- 172.21.15.19:0/1930487757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6cc19c770 con 0x7fd6cc103450 2024-04-25T12:04:17.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:17.016+0000 7fd6bb7fe700 1 -- 172.21.15.19:0/1930487757 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18440 (secure 0 0 0) 0x7fd6cc19c770 con 0x7fd6cc103450 2024-04-25T12:04:17.015 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:17.015 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:04:14.922729+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:04:17.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:17.018+0000 7fd6d0878700 1 -- 172.21.15.19:0/1930487757 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd6b40a3c40 msgr2=0x7fd6b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:17.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:17.018+0000 7fd6d0878700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd6b40a3c40 0x7fd6b40a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00b5c0 tx=0x7fd6bc005fb0 comp rx=0 tx=0).stop 2024-04-25T12:04:17.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:17.018+0000 7fd6d0878700 1 -- 172.21.15.19:0/1930487757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6cc103450 msgr2=0x7fd6cc19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:17.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:17.018+0000 7fd6d0878700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6cc103450 0x7fd6cc19b9a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000c970 tx=0x7fd6c000cd30 comp rx=0 tx=0).stop 2024-04-25T12:04:17.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:17.019+0000 7fd6d0878700 1 -- 172.21.15.19:0/1930487757 shutdown_connections 2024-04-25T12:04:17.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:17.019+0000 7fd6d0878700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd6b40a3c40 0x7fd6b40a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:17.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:17.019+0000 7fd6d0878700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6cc108a50 0x7fd6cc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:17.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:17.019+0000 7fd6d0878700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6cc102a90 0x7fd6cc19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:17.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:17.019+0000 7fd6d0878700 1 --2- 172.21.15.19:0/1930487757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd6cc103450 0x7fd6cc19b9a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:17.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:17.019+0000 7fd6d0878700 1 -- 172.21.15.19:0/1930487757 >> 172.21.15.19:0/1930487757 conn(0x7fd6cc079ba0 msgr2=0x7fd6cc101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:17.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:17.019+0000 7fd6d0878700 1 -- 172.21.15.19:0/1930487757 shutdown_connections 2024-04-25T12:04:17.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:17.019+0000 7fd6d0878700 1 -- 172.21.15.19:0/1930487757 wait complete. 2024-04-25T12:04:17.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:16 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:17.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:16 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:04:17.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:16 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:17.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:16 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:17.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:16 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:04:17.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:16 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:17.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:16 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:17.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:16 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:04:17.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:16 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:17.822 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:00:20.898809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T12:04:17.823 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T12:04:18.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:17 smithi019 ceph-mon[29213]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:18.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:17 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1930487757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:04:18.073 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:04:18.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:17 smithi169 ceph-mon[32710]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:18.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:17 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1930487757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:04:18.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:17 smithi150 ceph-mon[32250]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:18.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:17 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1930487757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:04:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.839+0000 7fc051dce700 1 -- 172.21.15.19:0/102650159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc04c103480 msgr2=0x7fc04c10f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.839+0000 7fc051dce700 1 --2- 172.21.15.19:0/102650159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc04c103480 0x7fc04c10f8d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc04400b0a0 tx=0x7fc04400b3b0 comp rx=0 tx=0).stop 2024-04-25T12:04:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.839+0000 7fc051dce700 1 -- 172.21.15.19:0/102650159 shutdown_connections 2024-04-25T12:04:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.839+0000 7fc051dce700 1 --2- 172.21.15.19:0/102650159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc04c103480 0x7fc04c10f8d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.839+0000 7fc051dce700 1 --2- 172.21.15.19:0/102650159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc04c108a70 0x7fc04c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.839+0000 7fc051dce700 1 --2- 172.21.15.19:0/102650159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc04c102ac0 0x7fc04c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.839+0000 7fc051dce700 1 -- 172.21.15.19:0/102650159 >> 172.21.15.19:0/102650159 conn(0x7fc04c079bb0 msgr2=0x7fc04c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:19.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.840+0000 7fc051dce700 1 -- 172.21.15.19:0/102650159 shutdown_connections 2024-04-25T12:04:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.840+0000 7fc051dce700 1 -- 172.21.15.19:0/102650159 wait complete. 2024-04-25T12:04:19.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.840+0000 7fc051dce700 1 Processor -- start 2024-04-25T12:04:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.840+0000 7fc051dce700 1 -- start start 2024-04-25T12:04:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.840+0000 7fc051dce700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc04c102ac0 0x7fc04c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:19.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.840+0000 7fc051dce700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc04c103480 0x7fc04c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.840+0000 7fc051dce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc04c108a70 0x7fc04c19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.840+0000 7fc051dce700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c112d60 con 0x7fc04c103480 2024-04-25T12:04:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.840+0000 7fc051dce700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c112be0 con 0x7fc04c108a70 2024-04-25T12:04:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.840+0000 7fc051dce700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c112ee0 con 0x7fc04c102ac0 2024-04-25T12:04:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc04affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc04c103480 0x7fc04c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc04affd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc04c103480 0x7fc04c199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59260/0 (socket says 172.21.15.19:59260) 2024-04-25T12:04:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc04affd700 1 -- 172.21.15.19:0/2764812970 learned_addr learned my addr 172.21.15.19:0/2764812970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc04bfff700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc04c108a70 0x7fc04c19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc04b7fe700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc04c102ac0 0x7fc04c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc04affd700 1 -- 172.21.15.19:0/2764812970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc04c102ac0 msgr2=0x7fc04c1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc04affd700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc04c102ac0 0x7fc04c1990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc04affd700 1 -- 172.21.15.19:0/2764812970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc04c108a70 msgr2=0x7fc04c19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc04affd700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc04c108a70 0x7fc04c19d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:19.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc04affd700 1 -- 172.21.15.19:0/2764812970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c19e0e0 con 0x7fc04c103480 2024-04-25T12:04:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc04bfff700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc04c108a70 0x7fc04c19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:04:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc04affd700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc04c103480 0x7fc04c199610 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc04000c920 tx=0x7fc04000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc048ff9700 1 -- 172.21.15.19:0/2764812970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040007b00 con 0x7fc04c103480 2024-04-25T12:04:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc051dce700 1 -- 172.21.15.19:0/2764812970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04c19e3d0 con 0x7fc04c103480 2024-04-25T12:04:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc048ff9700 1 -- 172.21.15.19:0/2764812970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc040004d00 con 0x7fc04c103480 2024-04-25T12:04:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc048ff9700 1 -- 172.21.15.19:0/2764812970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04000f450 con 0x7fc04c103480 2024-04-25T12:04:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.841+0000 7fc051dce700 1 -- 172.21.15.19:0/2764812970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04c1a5c50 con 0x7fc04c103480 2024-04-25T12:04:19.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.842+0000 7fc051dce700 1 -- 172.21.15.19:0/2764812970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04c069730 con 0x7fc04c103480 2024-04-25T12:04:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.843+0000 7fc048ff9700 1 -- 172.21.15.19:0/2764812970 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc040007ca0 con 0x7fc04c103480 2024-04-25T12:04:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.844+0000 7fc048ff9700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc0340a3c40 0x7fc0340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.844+0000 7fc04b7fe700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc0340a3c40 0x7fc0340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.844+0000 7fc048ff9700 1 -- 172.21.15.19:0/2764812970 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc0400cff40 con 0x7fc04c103480 2024-04-25T12:04:19.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.844+0000 7fc04b7fe700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc0340a3c40 0x7fc0340a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc03c005fd0 tx=0x7fc03c00af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:19.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:19.849+0000 7fc048ff9700 1 -- 172.21.15.19:0/2764812970 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc040014030 con 0x7fc04c103480 2024-04-25T12:04:20.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:20.053+0000 7fc051dce700 1 -- 172.21.15.19:0/2764812970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc04c1a5eb0 con 0x7fc04c103480 2024-04-25T12:04:20.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:20.054+0000 7fc048ff9700 1 -- 172.21.15.19:0/2764812970 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fc04009cdc0 con 0x7fc04c103480 2024-04-25T12:04:20.052 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T12:04:20.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:20.056+0000 7fc051dce700 1 -- 172.21.15.19:0/2764812970 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc0340a3c40 msgr2=0x7fc0340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:20.056+0000 7fc051dce700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc0340a3c40 0x7fc0340a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc03c005fd0 tx=0x7fc03c00af20 comp rx=0 tx=0).stop 2024-04-25T12:04:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:20.056+0000 7fc051dce700 1 -- 172.21.15.19:0/2764812970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc04c103480 msgr2=0x7fc04c199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:20.056+0000 7fc051dce700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc04c103480 0x7fc04c199610 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc04000c920 tx=0x7fc04000cce0 comp rx=0 tx=0).stop 2024-04-25T12:04:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:20.056+0000 7fc051dce700 1 -- 172.21.15.19:0/2764812970 shutdown_connections 2024-04-25T12:04:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:20.056+0000 7fc051dce700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc0340a3c40 0x7fc0340a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:20.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:20.056+0000 7fc051dce700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc04c108a70 0x7fc04c19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:20.056+0000 7fc051dce700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc04c102ac0 0x7fc04c1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:20.056+0000 7fc051dce700 1 --2- 172.21.15.19:0/2764812970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc04c103480 0x7fc04c199610 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:20.056+0000 7fc051dce700 1 -- 172.21.15.19:0/2764812970 >> 172.21.15.19:0/2764812970 conn(0x7fc04c079bb0 msgr2=0x7fc04c101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:20.056+0000 7fc051dce700 1 -- 172.21.15.19:0/2764812970 shutdown_connections 2024-04-25T12:04:20.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:20.056+0000 7fc051dce700 1 -- 172.21.15.19:0/2764812970 wait complete. 2024-04-25T12:04:20.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:19 smithi019 ceph-mon[29213]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:20.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:19 smithi169 ceph-mon[32710]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:20.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:19 smithi150 ceph-mon[32250]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:20.745 INFO:tasks.cephadm:Adding mds.a on smithi019 2024-04-25T12:04:20.745 INFO:tasks.cephadm:Adding mds.d on smithi019 2024-04-25T12:04:20.745 INFO:tasks.cephadm:Adding mds.g on smithi019 2024-04-25T12:04:20.745 INFO:tasks.cephadm:Adding mds.j on smithi019 2024-04-25T12:04:20.746 INFO:tasks.cephadm:Adding mds.b on smithi150 2024-04-25T12:04:20.746 INFO:tasks.cephadm:Adding mds.e on smithi150 2024-04-25T12:04:20.746 INFO:tasks.cephadm:Adding mds.h on smithi150 2024-04-25T12:04:20.746 INFO:tasks.cephadm:Adding mds.k on smithi150 2024-04-25T12:04:20.746 INFO:tasks.cephadm:Adding mds.c on smithi169 2024-04-25T12:04:20.746 INFO:tasks.cephadm:Adding mds.f on smithi169 2024-04-25T12:04:20.746 INFO:tasks.cephadm:Adding mds.i on smithi169 2024-04-25T12:04:20.746 INFO:tasks.cephadm:Adding mds.l on smithi169 2024-04-25T12:04:20.746 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l' 2024-04-25T12:04:20.999 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:04:21.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:20 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2764812970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:04:21.121 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:20 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2764812970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:04:21.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:20 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2764812970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:04:22.249 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:21 smithi169 ceph-mon[32710]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:22.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:21 smithi019 ceph-mon[29213]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:22.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:21 smithi150 ceph-mon[32250]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:22.606 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.603+0000 7fde697ac700 1 -- 172.21.15.169:0/4203639788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde64102ba0 msgr2=0x7fde64103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:22.606 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.603+0000 7fde697ac700 1 --2- 172.21.15.169:0/4203639788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde64102ba0 0x7fde64103000 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009990 tx=0x7fde4c009ca0 comp rx=0 tx=0).stop 2024-04-25T12:04:22.606 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.603+0000 7fde697ac700 1 -- 172.21.15.169:0/4203639788 shutdown_connections 2024-04-25T12:04:22.606 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.603+0000 7fde697ac700 1 --2- 172.21.15.169:0/4203639788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde641019a0 0x7fde64101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:22.606 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.603+0000 7fde697ac700 1 --2- 172.21.15.169:0/4203639788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde64103540 0x7fde64109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:22.606 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.603+0000 7fde697ac700 1 --2- 172.21.15.169:0/4203639788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde64102ba0 0x7fde64103000 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:22.606 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.603+0000 7fde697ac700 1 -- 172.21.15.169:0/4203639788 >> 172.21.15.169:0/4203639788 conn(0x7fde640fcf40 msgr2=0x7fde640ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:22.606 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.603+0000 7fde697ac700 1 -- 172.21.15.169:0/4203639788 shutdown_connections 2024-04-25T12:04:22.606 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.603+0000 7fde697ac700 1 -- 172.21.15.169:0/4203639788 wait complete. 2024-04-25T12:04:22.607 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.603+0000 7fde697ac700 1 Processor -- start 2024-04-25T12:04:22.607 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde697ac700 1 -- start start 2024-04-25T12:04:22.607 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde697ac700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde641019a0 0x7fde6406b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:22.607 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde697ac700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde64102ba0 0x7fde6419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:22.607 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde697ac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde64103540 0x7fde641a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:22.607 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde697ac700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde6410d5d0 con 0x7fde64102ba0 2024-04-25T12:04:22.607 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde697ac700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fde6410d450 con 0x7fde64103540 2024-04-25T12:04:22.607 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde697ac700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fde6410d750 con 0x7fde641019a0 2024-04-25T12:04:22.607 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde62ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde641019a0 0x7fde6406b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:22.607 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde62ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde641019a0 0x7fde6406b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:48834/0 (socket says 172.21.15.169:48834) 2024-04-25T12:04:22.607 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde62ffd700 1 -- 172.21.15.169:0/2917911864 learned_addr learned my addr 172.21.15.169:0/2917911864 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-04-25T12:04:22.607 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde5a7fc700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde64102ba0 0x7fde6419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:22.607 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde637fe700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde64103540 0x7fde641a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:22.607 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde62ffd700 1 -- 172.21.15.169:0/2917911864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde64103540 msgr2=0x7fde641a0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde62ffd700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde64103540 0x7fde641a0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde62ffd700 1 -- 172.21.15.169:0/2917911864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde64102ba0 msgr2=0x7fde6419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde62ffd700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde64102ba0 0x7fde6419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde62ffd700 1 -- 172.21.15.169:0/2917911864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde641a0930 con 0x7fde641019a0 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde637fe700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde64103540 0x7fde641a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.604+0000 7fde62ffd700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde641019a0 0x7fde6406b280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fde54007e00 tx=0x7fde5400a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.605+0000 7fde60ff9700 1 -- 172.21.15.169:0/2917911864 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54016020 con 0x7fde641019a0 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.605+0000 7fde60ff9700 1 -- 172.21.15.169:0/2917911864 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fde5400ad40 con 0x7fde641019a0 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.605+0000 7fde697ac700 1 -- 172.21.15.169:0/2917911864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde641a0c20 con 0x7fde641019a0 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.605+0000 7fde60ff9700 1 -- 172.21.15.169:0/2917911864 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54011700 con 0x7fde641019a0 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.605+0000 7fde697ac700 1 -- 172.21.15.169:0/2917911864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde641a1160 con 0x7fde641019a0 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.606+0000 7fde697ac700 1 -- 172.21.15.169:0/2917911864 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde64069730 con 0x7fde641019a0 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.607+0000 7fde60ff9700 1 -- 172.21.15.169:0/2917911864 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fde54019020 con 0x7fde641019a0 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.607+0000 7fde60ff9700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fde500a3c40 0x7fde500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:22.608 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.608+0000 7fde60ff9700 1 -- 172.21.15.169:0/2917911864 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fde540cfbd0 con 0x7fde641019a0 2024-04-25T12:04:22.609 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.608+0000 7fde5a7fc700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fde500a3c40 0x7fde500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:22.609 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.608+0000 7fde5a7fc700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fde500a3c40 0x7fde500a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fde6406b800 tx=0x7fde4c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:22.610 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.612+0000 7fde60ff9700 1 -- 172.21.15.169:0/2917911864 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde54020030 con 0x7fde641019a0 2024-04-25T12:04:22.816 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.819+0000 7fde697ac700 1 -- 172.21.15.169:0/2917911864 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}) -- 0x7fde64063080 con 0x7fde500a3c40 2024-04-25T12:04:22.824 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.827+0000 7fde60ff9700 1 -- 172.21.15.169:0/2917911864 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fde64063080 con 0x7fde500a3c40 2024-04-25T12:04:22.824 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mds.all update... 2024-04-25T12:04:22.827 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.829+0000 7fde697ac700 1 -- 172.21.15.169:0/2917911864 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fde500a3c40 msgr2=0x7fde500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:22.827 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.829+0000 7fde697ac700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fde500a3c40 0x7fde500a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fde6406b800 tx=0x7fde4c005fd0 comp rx=0 tx=0).stop 2024-04-25T12:04:22.827 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.829+0000 7fde697ac700 1 -- 172.21.15.169:0/2917911864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde641019a0 msgr2=0x7fde6406b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:22.827 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.829+0000 7fde697ac700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde641019a0 0x7fde6406b280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fde54007e00 tx=0x7fde5400a340 comp rx=0 tx=0).stop 2024-04-25T12:04:22.827 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.829+0000 7fde697ac700 1 -- 172.21.15.169:0/2917911864 shutdown_connections 2024-04-25T12:04:22.827 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.829+0000 7fde697ac700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fde500a3c40 0x7fde500a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:22.827 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.829+0000 7fde697ac700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde64103540 0x7fde641a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:22.827 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.829+0000 7fde697ac700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde641019a0 0x7fde6406b280 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:22.827 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.829+0000 7fde697ac700 1 --2- 172.21.15.169:0/2917911864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde64102ba0 0x7fde6419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:22.827 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.829+0000 7fde697ac700 1 -- 172.21.15.169:0/2917911864 >> 172.21.15.169:0/2917911864 conn(0x7fde640fcf40 msgr2=0x7fde64105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:22.827 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.829+0000 7fde697ac700 1 -- 172.21.15.169:0/2917911864 shutdown_connections 2024-04-25T12:04:22.827 INFO:teuthology.orchestra.run.smithi169.stderr:2024-04-25T12:04:22.829+0000 7fde697ac700 1 -- 172.21.15.169:0/2917911864 wait complete. 2024-04-25T12:04:23.487 DEBUG:teuthology.orchestra.run.smithi019:mds.a> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T12:04:23.491 DEBUG:teuthology.orchestra.run.smithi019:mds.d> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T12:04:23.534 DEBUG:teuthology.orchestra.run.smithi019:mds.g> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T12:04:23.577 DEBUG:teuthology.orchestra.run.smithi019:mds.j> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T12:04:23.621 DEBUG:teuthology.orchestra.run.smithi150:mds.b> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T12:04:23.624 DEBUG:teuthology.orchestra.run.smithi150:mds.e> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T12:04:23.668 DEBUG:teuthology.orchestra.run.smithi150:mds.h> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T12:04:23.710 DEBUG:teuthology.orchestra.run.smithi150:mds.k> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T12:04:23.753 DEBUG:teuthology.orchestra.run.smithi169:mds.c> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T12:04:23.756 DEBUG:teuthology.orchestra.run.smithi169:mds.f> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T12:04:23.800 DEBUG:teuthology.orchestra.run.smithi169:mds.i> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T12:04:23.842 DEBUG:teuthology.orchestra.run.smithi169:mds.l> sudo journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T12:04:23.885 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T12:04:23.885 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:04:23.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:23 smithi150 ceph-mon[32250]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:04:23.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:23 smithi150 ceph-mon[32250]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T12:04:23.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:23.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:04:23.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:23.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:04:23.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:23.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:23.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:23.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:23.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:23 smithi150 ceph-mon[32250]: Deploying daemon mds.l on smithi169 2024-04-25T12:04:23.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:23 smithi150 ceph-mon[32250]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:24.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.002+0000 7f57e58e8640 1 -- 172.21.15.19:0/2755586328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0102a30 msgr2=0x7f57e0104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.002+0000 7f57e58e8640 1 --2- 172.21.15.19:0/2755586328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0102a30 0x7f57e0104e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009900 tx=0x7f57d002a410 comp rx=0 tx=0).stop 2024-04-25T12:04:24.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.002+0000 7f57e58e8640 1 -- 172.21.15.19:0/2755586328 shutdown_connections 2024-04-25T12:04:24.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.002+0000 7f57e58e8640 1 --2- 172.21.15.19:0/2755586328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57e0107c90 0x7f57e010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.002+0000 7f57e58e8640 1 --2- 172.21.15.19:0/2755586328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e0105360 0x7f57e0107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.002+0000 7f57e58e8640 1 --2- 172.21.15.19:0/2755586328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0102a30 0x7f57e0104e20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.002+0000 7f57e58e8640 1 -- 172.21.15.19:0/2755586328 >> 172.21.15.19:0/2755586328 conn(0x7f57e00fc440 msgr2=0x7f57e00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:24.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.002+0000 7f57e58e8640 1 -- 172.21.15.19:0/2755586328 shutdown_connections 2024-04-25T12:04:24.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.002+0000 7f57e58e8640 1 -- 172.21.15.19:0/2755586328 wait complete. 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.002+0000 7f57e58e8640 1 Processor -- start 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.002+0000 7f57e58e8640 1 -- start start 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57e58e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0102a30 0x7f57e0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57e58e8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57e0105360 0x7f57e01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57e58e8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e0107c90 0x7f57e019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57e58e8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f57e010dc10 con 0x7f57e0102a30 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57e58e8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f57e010da90 con 0x7f57e0107c90 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57e58e8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f57e010dd90 con 0x7f57e0105360 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0102a30 0x7f57e0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0102a30 0x7f57e0198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39540/0 (socket says 172.21.15.19:39540) 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57deffd640 1 -- 172.21.15.19:0/1965873661 learned_addr learned my addr 172.21.15.19:0/1965873661 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57de7fc640 1 --2- 172.21.15.19:0/1965873661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57e0105360 0x7f57e01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57deffd640 1 -- 172.21.15.19:0/1965873661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57e0105360 msgr2=0x7f57e01987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57deffd640 1 --2- 172.21.15.19:0/1965873661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57e0105360 0x7f57e01987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57deffd640 1 -- 172.21.15.19:0/1965873661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e0107c90 msgr2=0x7f57e019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57df7fe640 1 --2- 172.21.15.19:0/1965873661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e0107c90 0x7f57e019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:24.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57deffd640 1 --2- 172.21.15.19:0/1965873661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e0107c90 0x7f57e019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57deffd640 1 -- 172.21.15.19:0/1965873661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e019fe80 con 0x7f57e0102a30 2024-04-25T12:04:24.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57deffd640 1 --2- 172.21.15.19:0/1965873661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0102a30 0x7f57e0198260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f57d0005940 tx=0x7f57d00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:24.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57e48e6640 1 -- 172.21.15.19:0/1965873661 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d003f030 con 0x7f57e0102a30 2024-04-25T12:04:24.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57e48e6640 1 -- 172.21.15.19:0/1965873661 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f57d0003cd0 con 0x7f57e0102a30 2024-04-25T12:04:24.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57e58e8640 1 -- 172.21.15.19:0/1965873661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e01a0110 con 0x7f57e0102a30 2024-04-25T12:04:24.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.003+0000 7f57e48e6640 1 -- 172.21.15.19:0/1965873661 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0035070 con 0x7f57e0102a30 2024-04-25T12:04:24.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.004+0000 7f57e58e8640 1 -- 172.21.15.19:0/1965873661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e01a0650 con 0x7f57e0102a30 2024-04-25T12:04:24.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.005+0000 7f57e58e8640 1 -- 172.21.15.19:0/1965873661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e00fdf30 con 0x7f57e0102a30 2024-04-25T12:04:24.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.006+0000 7f57e48e6640 1 -- 172.21.15.19:0/1965873661 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f57d0042030 con 0x7f57e0102a30 2024-04-25T12:04:24.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.006+0000 7f57e48e6640 1 --2- 172.21.15.19:0/1965873661 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f57c80a3c30 0x7f57c80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:24.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.006+0000 7f57de7fc640 1 --2- 172.21.15.19:0/1965873661 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f57c80a3c30 0x7f57c80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:24.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.007+0000 7f57e48e6640 1 -- 172.21.15.19:0/1965873661 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f57d00f04f0 con 0x7f57e0102a30 2024-04-25T12:04:24.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.007+0000 7f57de7fc640 1 --2- 172.21.15.19:0/1965873661 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f57c80a3c30 0x7f57c80a60f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f57e0199770 tx=0x7f57d400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:24.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.011+0000 7f57e48e6640 1 -- 172.21.15.19:0/1965873661 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57d00bd370 con 0x7f57e0102a30 2024-04-25T12:04:24.022 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:23 smithi169 ceph-mon[32710]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:04:24.022 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:23 smithi169 ceph-mon[32710]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T12:04:24.023 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:24.023 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:04:24.023 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:24.023 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:04:24.023 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:24.023 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:24.023 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:24.023 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:24.023 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:23 smithi169 ceph-mon[32710]: Deploying daemon mds.l on smithi169 2024-04-25T12:04:24.023 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:23 smithi169 ceph-mon[32710]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:24.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.201+0000 7f57e58e8640 1 -- 172.21.15.19:0/1965873661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57e0062b50 con 0x7f57e0102a30 2024-04-25T12:04:24.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.202+0000 7f57e48e6640 1 -- 172.21.15.19:0/1965873661 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18440 (secure 0 0 0) 0x7f57d00bcc70 con 0x7f57e0102a30 2024-04-25T12:04:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:24.200 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:04:14.922729+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:04:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.203+0000 7f57e58e8640 1 -- 172.21.15.19:0/1965873661 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f57c80a3c30 msgr2=0x7f57c80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.203+0000 7f57e58e8640 1 --2- 172.21.15.19:0/1965873661 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f57c80a3c30 0x7f57c80a60f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f57e0199770 tx=0x7f57d400a2d0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.204+0000 7f57e58e8640 1 -- 172.21.15.19:0/1965873661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0102a30 msgr2=0x7f57e0198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.204+0000 7f57e58e8640 1 --2- 172.21.15.19:0/1965873661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0102a30 0x7f57e0198260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f57d0005940 tx=0x7f57d00043b0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.204+0000 7f57e58e8640 1 -- 172.21.15.19:0/1965873661 shutdown_connections 2024-04-25T12:04:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.204+0000 7f57e58e8640 1 --2- 172.21.15.19:0/1965873661 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f57c80a3c30 0x7f57c80a60f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.204+0000 7f57e58e8640 1 --2- 172.21.15.19:0/1965873661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57e0107c90 0x7f57e019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.204+0000 7f57e58e8640 1 --2- 172.21.15.19:0/1965873661 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57e0105360 0x7f57e01987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.204+0000 7f57e58e8640 1 --2- 172.21.15.19:0/1965873661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0102a30 0x7f57e0198260 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.204+0000 7f57e58e8640 1 -- 172.21.15.19:0/1965873661 >> 172.21.15.19:0/1965873661 conn(0x7f57e00fc440 msgr2=0x7f57e0103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:24.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.204+0000 7f57e58e8640 1 -- 172.21.15.19:0/1965873661 shutdown_connections 2024-04-25T12:04:24.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.204+0000 7f57e58e8640 1 -- 172.21.15.19:0/1965873661 wait complete. 2024-04-25T12:04:24.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:23 smithi019 ceph-mon[29213]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:04:24.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:23 smithi019 ceph-mon[29213]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T12:04:24.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:24.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:04:24.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:24.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:04:24.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:24.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:24.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:24.212 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:24.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:23 smithi019 ceph-mon[29213]: Deploying daemon mds.l on smithi169 2024-04-25T12:04:24.213 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:23 smithi019 ceph-mon[29213]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:24.214 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:00:20.898809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T12:04:24.214 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:04:24.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.321+0000 7ff994bd6640 1 -- 172.21.15.19:0/1678487321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff990100e60 msgr2=0x7ff990101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.321+0000 7ff994bd6640 1 --2- 172.21.15.19:0/1678487321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff990100e60 0x7ff990101280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff980009900 tx=0x7ff98002a410 comp rx=0 tx=0).stop 2024-04-25T12:04:24.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.321+0000 7ff994bd6640 1 -- 172.21.15.19:0/1678487321 shutdown_connections 2024-04-25T12:04:24.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.321+0000 7ff994bd6640 1 --2- 172.21.15.19:0/1678487321 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff990102a00 0x7ff990109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.321+0000 7ff994bd6640 1 --2- 172.21.15.19:0/1678487321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff990102040 0x7ff9901024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.321+0000 7ff994bd6640 1 --2- 172.21.15.19:0/1678487321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff990100e60 0x7ff990101280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.321+0000 7ff994bd6640 1 -- 172.21.15.19:0/1678487321 >> 172.21.15.19:0/1678487321 conn(0x7ff9900fc440 msgr2=0x7ff9900fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.321+0000 7ff994bd6640 1 -- 172.21.15.19:0/1678487321 shutdown_connections 2024-04-25T12:04:24.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.321+0000 7ff994bd6640 1 -- 172.21.15.19:0/1678487321 wait complete. 2024-04-25T12:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.321+0000 7ff994bd6640 1 Processor -- start 2024-04-25T12:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff994bd6640 1 -- start start 2024-04-25T12:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff994bd6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff990100e60 0x7ff990197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:24.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff994bd6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff990102040 0x7ff990197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff994bd6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff990102a00 0x7ff99019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff994bd6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff99010cb30 con 0x7ff990100e60 2024-04-25T12:04:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff994bd6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff99010c9b0 con 0x7ff990102a00 2024-04-25T12:04:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff994bd6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff99010ccb0 con 0x7ff990102040 2024-04-25T12:04:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff98e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff990100e60 0x7ff990197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff98e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff990100e60 0x7ff990197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39554/0 (socket says 172.21.15.19:39554) 2024-04-25T12:04:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff98e59c640 1 -- 172.21.15.19:0/3115679301 learned_addr learned my addr 172.21.15.19:0/3115679301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff98ed9d640 1 --2- 172.21.15.19:0/3115679301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff990102a00 0x7ff99019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff98dd9b640 1 --2- 172.21.15.19:0/3115679301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff990102040 0x7ff990197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff98e59c640 1 -- 172.21.15.19:0/3115679301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff990102040 msgr2=0x7ff990197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff98e59c640 1 --2- 172.21.15.19:0/3115679301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff990102040 0x7ff990197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff98e59c640 1 -- 172.21.15.19:0/3115679301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff990102a00 msgr2=0x7ff99019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff98e59c640 1 --2- 172.21.15.19:0/3115679301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff990102a00 0x7ff99019e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff98e59c640 1 -- 172.21.15.19:0/3115679301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99019ee20 con 0x7ff990100e60 2024-04-25T12:04:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff98e59c640 1 --2- 172.21.15.19:0/3115679301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff990100e60 0x7ff990197200 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff980006010 tx=0x7ff9800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff97f7fe640 1 -- 172.21.15.19:0/3115679301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98003f030 con 0x7ff990100e60 2024-04-25T12:04:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff97f7fe640 1 -- 172.21.15.19:0/3115679301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff980003cd0 con 0x7ff990100e60 2024-04-25T12:04:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.322+0000 7ff994bd6640 1 -- 172.21.15.19:0/3115679301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99019f0b0 con 0x7ff990100e60 2024-04-25T12:04:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.323+0000 7ff97f7fe640 1 -- 172.21.15.19:0/3115679301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980035070 con 0x7ff990100e60 2024-04-25T12:04:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.323+0000 7ff994bd6640 1 -- 172.21.15.19:0/3115679301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99019f5a0 con 0x7ff990100e60 2024-04-25T12:04:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.323+0000 7ff994bd6640 1 -- 172.21.15.19:0/3115679301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff990108270 con 0x7ff990100e60 2024-04-25T12:04:24.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.325+0000 7ff97f7fe640 1 -- 172.21.15.19:0/3115679301 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff980042030 con 0x7ff990100e60 2024-04-25T12:04:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.325+0000 7ff97f7fe640 1 --2- 172.21.15.19:0/3115679301 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff9780a3bb0 0x7ff9780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.325+0000 7ff98dd9b640 1 --2- 172.21.15.19:0/3115679301 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff9780a3bb0 0x7ff9780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.325+0000 7ff97f7fe640 1 -- 172.21.15.19:0/3115679301 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff9800f0420 con 0x7ff990100e60 2024-04-25T12:04:24.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.326+0000 7ff98dd9b640 1 --2- 172.21.15.19:0/3115679301 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff9780a3bb0 0x7ff9780a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff990198710 tx=0x7ff98400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:24.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.330+0000 7ff97f7fe640 1 -- 172.21.15.19:0/3115679301 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff9800bcc50 con 0x7ff990100e60 2024-04-25T12:04:24.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.516+0000 7ff994bd6640 1 -- 172.21.15.19:0/3115679301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff990062b50 con 0x7ff990100e60 2024-04-25T12:04:24.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.516+0000 7ff97f7fe640 1 -- 172.21.15.19:0/3115679301 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7ff9800c0f50 con 0x7ff990100e60 2024-04-25T12:04:24.514 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T12:04:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.518+0000 7ff994bd6640 1 -- 172.21.15.19:0/3115679301 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff9780a3bb0 msgr2=0x7ff9780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.518+0000 7ff994bd6640 1 --2- 172.21.15.19:0/3115679301 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff9780a3bb0 0x7ff9780a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff990198710 tx=0x7ff98400b390 comp rx=0 tx=0).stop 2024-04-25T12:04:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.518+0000 7ff994bd6640 1 -- 172.21.15.19:0/3115679301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff990100e60 msgr2=0x7ff990197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.518+0000 7ff994bd6640 1 --2- 172.21.15.19:0/3115679301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff990100e60 0x7ff990197200 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff980006010 tx=0x7ff9800042f0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.518+0000 7ff994bd6640 1 -- 172.21.15.19:0/3115679301 shutdown_connections 2024-04-25T12:04:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.518+0000 7ff994bd6640 1 --2- 172.21.15.19:0/3115679301 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff9780a3bb0 0x7ff9780a6070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.518+0000 7ff994bd6640 1 --2- 172.21.15.19:0/3115679301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff990102a00 0x7ff99019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.518+0000 7ff994bd6640 1 --2- 172.21.15.19:0/3115679301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff990102040 0x7ff990197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.518+0000 7ff994bd6640 1 --2- 172.21.15.19:0/3115679301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff990100e60 0x7ff990197200 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.518+0000 7ff994bd6640 1 -- 172.21.15.19:0/3115679301 >> 172.21.15.19:0/3115679301 conn(0x7ff9900fc440 msgr2=0x7ff990105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.518+0000 7ff994bd6640 1 -- 172.21.15.19:0/3115679301 shutdown_connections 2024-04-25T12:04:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.518+0000 7ff994bd6640 1 -- 172.21.15.19:0/3115679301 wait complete. 2024-04-25T12:04:24.527 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:24.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.631+0000 7fd4ada31640 1 -- 172.21.15.19:0/2774046785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a8105340 msgr2=0x7fd4a8107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.631+0000 7fd4ada31640 1 --2- 172.21.15.19:0/2774046785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a8105340 0x7fd4a8107730 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd490009900 tx=0x7fd49002a410 comp rx=0 tx=0).stop 2024-04-25T12:04:24.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.631+0000 7fd4ada31640 1 -- 172.21.15.19:0/2774046785 shutdown_connections 2024-04-25T12:04:24.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.631+0000 7fd4ada31640 1 --2- 172.21.15.19:0/2774046785 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a8107c70 0x7fd4a810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.631+0000 7fd4ada31640 1 --2- 172.21.15.19:0/2774046785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a8105340 0x7fd4a8107730 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.631+0000 7fd4ada31640 1 --2- 172.21.15.19:0/2774046785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a8102a10 0x7fd4a8104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.631+0000 7fd4ada31640 1 -- 172.21.15.19:0/2774046785 >> 172.21.15.19:0/2774046785 conn(0x7fd4a80fc440 msgr2=0x7fd4a80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.631+0000 7fd4ada31640 1 -- 172.21.15.19:0/2774046785 shutdown_connections 2024-04-25T12:04:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.631+0000 7fd4ada31640 1 -- 172.21.15.19:0/2774046785 wait complete. 2024-04-25T12:04:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4ada31640 1 Processor -- start 2024-04-25T12:04:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4ada31640 1 -- start start 2024-04-25T12:04:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4ada31640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a8102a10 0x7fd4a81982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4ada31640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a8105340 0x7fd4a8198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4ada31640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a8107c70 0x7fd4a819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4ada31640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810dbf0 con 0x7fd4a8107c70 2024-04-25T12:04:24.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4ada31640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810da70 con 0x7fd4a8102a10 2024-04-25T12:04:24.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4ada31640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810dd70 con 0x7fd4a8105340 2024-04-25T12:04:24.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4a77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a8107c70 0x7fd4a819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:24.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4a77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a8107c70 0x7fd4a819f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39586/0 (socket says 172.21.15.19:39586) 2024-04-25T12:04:24.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4a77fe640 1 -- 172.21.15.19:0/1791907273 learned_addr learned my addr 172.21.15.19:0/1791907273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:24.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd49ffff640 1 --2- 172.21.15.19:0/1791907273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a8105340 0x7fd4a8198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:24.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4a6ffd640 1 --2- 172.21.15.19:0/1791907273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a8102a10 0x7fd4a81982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:24.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4a77fe640 1 -- 172.21.15.19:0/1791907273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a8105340 msgr2=0x7fd4a8198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4a77fe640 1 --2- 172.21.15.19:0/1791907273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a8105340 0x7fd4a8198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4a77fe640 1 -- 172.21.15.19:0/1791907273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a8102a10 msgr2=0x7fd4a81982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4a77fe640 1 --2- 172.21.15.19:0/1791907273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a8102a10 0x7fd4a81982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.632+0000 7fd4a77fe640 1 -- 172.21.15.19:0/1791907273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a819fee0 con 0x7fd4a8107c70 2024-04-25T12:04:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.633+0000 7fd4a77fe640 1 --2- 172.21.15.19:0/1791907273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a8107c70 0x7fd4a819f870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000eb10 tx=0x7fd4a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.633+0000 7fd4aca2f640 1 -- 172.21.15.19:0/1791907273 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0004b20 con 0x7fd4a8107c70 2024-04-25T12:04:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.633+0000 7fd4aca2f640 1 -- 172.21.15.19:0/1791907273 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd4a0004cc0 con 0x7fd4a8107c70 2024-04-25T12:04:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.633+0000 7fd4aca2f640 1 -- 172.21.15.19:0/1791907273 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0010720 con 0x7fd4a8107c70 2024-04-25T12:04:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.633+0000 7fd4ada31640 1 -- 172.21.15.19:0/1791907273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a81a01d0 con 0x7fd4a8107c70 2024-04-25T12:04:24.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.633+0000 7fd4ada31640 1 -- 172.21.15.19:0/1791907273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a81a0740 con 0x7fd4a8107c70 2024-04-25T12:04:24.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.634+0000 7fd4ada31640 1 -- 172.21.15.19:0/1791907273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a80fdf30 con 0x7fd4a8107c70 2024-04-25T12:04:24.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.635+0000 7fd4aca2f640 1 -- 172.21.15.19:0/1791907273 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd4a0005000 con 0x7fd4a8107c70 2024-04-25T12:04:24.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.636+0000 7fd4aca2f640 1 --2- 172.21.15.19:0/1791907273 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd4940a3c30 0x7fd4940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:24.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.636+0000 7fd4a6ffd640 1 --2- 172.21.15.19:0/1791907273 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd4940a3c30 0x7fd4940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:24.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.636+0000 7fd4aca2f640 1 -- 172.21.15.19:0/1791907273 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd4a00d0710 con 0x7fd4a8107c70 2024-04-25T12:04:24.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.636+0000 7fd4a6ffd640 1 --2- 172.21.15.19:0/1791907273 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd4940a3c30 0x7fd4940a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd498005fd0 tx=0x7fd498005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:24.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.640+0000 7fd4aca2f640 1 -- 172.21.15.19:0/1791907273 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4a0053ff0 con 0x7fd4a8107c70 2024-04-25T12:04:24.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.865+0000 7fd4ada31640 1 -- 172.21.15.19:0/1791907273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4a802d5c0 con 0x7fd4a8107c70 2024-04-25T12:04:24.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.865+0000 7fd4aca2f640 1 -- 172.21.15.19:0/1791907273 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fd4a009d570 con 0x7fd4a8107c70 2024-04-25T12:04:24.864 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:24.864 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T12:04:24.864 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 1 2024-04-25T12:04:24.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.867+0000 7fd4ada31640 1 -- 172.21.15.19:0/1791907273 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd4940a3c30 msgr2=0x7fd4940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.867+0000 7fd4ada31640 1 --2- 172.21.15.19:0/1791907273 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd4940a3c30 0x7fd4940a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd498005fd0 tx=0x7fd498005850 comp rx=0 tx=0).stop 2024-04-25T12:04:24.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.867+0000 7fd4ada31640 1 -- 172.21.15.19:0/1791907273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a8107c70 msgr2=0x7fd4a819f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:24.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.867+0000 7fd4ada31640 1 --2- 172.21.15.19:0/1791907273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a8107c70 0x7fd4a819f870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000eb10 tx=0x7fd4a000c2d0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.867+0000 7fd4ada31640 1 -- 172.21.15.19:0/1791907273 shutdown_connections 2024-04-25T12:04:24.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.867+0000 7fd4ada31640 1 --2- 172.21.15.19:0/1791907273 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd4940a3c30 0x7fd4940a60f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.867+0000 7fd4ada31640 1 --2- 172.21.15.19:0/1791907273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a8107c70 0x7fd4a819f870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.867+0000 7fd4ada31640 1 --2- 172.21.15.19:0/1791907273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a8105340 0x7fd4a8198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.867+0000 7fd4ada31640 1 --2- 172.21.15.19:0/1791907273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a8102a10 0x7fd4a81982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:24.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.867+0000 7fd4ada31640 1 -- 172.21.15.19:0/1791907273 >> 172.21.15.19:0/1791907273 conn(0x7fd4a80fc440 msgr2=0x7fd4a8103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:24.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.868+0000 7fd4ada31640 1 -- 172.21.15.19:0/1791907273 shutdown_connections 2024-04-25T12:04:24.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:24.868+0000 7fd4ada31640 1 -- 172.21.15.19:0/1791907273 wait complete. 2024-04-25T12:04:24.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:24 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1965873661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:04:24.939 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:24 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3115679301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:04:24.940 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:24 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1791907273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:25.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:24 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1965873661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:04:25.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:24 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3115679301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:04:25.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:24 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1791907273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:25.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:24 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1965873661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:04:25.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:24 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3115679301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:04:25.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:24 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1791907273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:25.334 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:04:25 smithi169 systemd[1]: Starting Ceph mds.l for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:04:25.599 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:04:25 smithi169 podman[63526]: 2024-04-25 12:04:25.240284037 +0000 UTC m=+0.017868626 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:04:25.600 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:04:25 smithi169 podman[63526]: 2024-04-25 12:04:25.349819623 +0000 UTC m=+0.127404245 container create 49e057e4f52087136c7c249214e8e0ac1caa03b1cbff85bfde28f19f03273be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-04-25T12:04:25.600 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:04:25 smithi169 podman[63526]: 2024-04-25 12:04:25.602623199 +0000 UTC m=+0.380207776 container init 49e057e4f52087136c7c249214e8e0ac1caa03b1cbff85bfde28f19f03273be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T12:04:25.905 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:04:25 smithi169 podman[63526]: 2024-04-25 12:04:25.605188843 +0000 UTC m=+0.382773421 container start 49e057e4f52087136c7c249214e8e0ac1caa03b1cbff85bfde28f19f03273be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T12:04:25.905 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:04:25 smithi169 ceph-mds[63545]: -- 172.21.15.169:0/3493420301 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563cd638ba40 con 0x563cd635ec00 2024-04-25T12:04:25.905 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:04:25 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-l[63541]: starting mds.l at 2024-04-25T12:04:25.905 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:04:25 smithi169 bash[63526]: 49e057e4f52087136c7c249214e8e0ac1caa03b1cbff85bfde28f19f03273be1 2024-04-25T12:04:25.906 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:04:25 smithi169 systemd[1]: Started Ceph mds.l for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T12:04:26.650 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:26 smithi150 ceph-mon[32250]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:26.650 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:26 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.169:6832/2217300694,v1:172.21.15.169:6833/2217300694] up:boot 2024-04-25T12:04:26.650 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:26 smithi150 ceph-mon[32250]: fsmap 1 up:standby 2024-04-25T12:04:26.650 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T12:04:26.650 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:26.651 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:26.651 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:26.651 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:26.651 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:26.651 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:26 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:26.651 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:26 smithi150 ceph-mon[32250]: Deploying daemon mds.k on smithi150 2024-04-25T12:04:26.877 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:26 smithi019 ceph-mon[29213]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:26 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.169:6832/2217300694,v1:172.21.15.169:6833/2217300694] up:boot 2024-04-25T12:04:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:26 smithi019 ceph-mon[29213]: fsmap 1 up:standby 2024-04-25T12:04:26.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T12:04:26.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:26.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:26.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:26.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:26.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:26.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:26 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:26.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:26 smithi019 ceph-mon[29213]: Deploying daemon mds.k on smithi150 2024-04-25T12:04:26.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:26 smithi169 ceph-mon[32710]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:26.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:26 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.169:6832/2217300694,v1:172.21.15.169:6833/2217300694] up:boot 2024-04-25T12:04:26.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:26 smithi169 ceph-mon[32710]: fsmap 1 up:standby 2024-04-25T12:04:26.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T12:04:26.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:26.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:26.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:26.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:26.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:26.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:26 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:26.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:26 smithi169 ceph-mon[32710]: Deploying daemon mds.k on smithi150 2024-04-25T12:04:26.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.995+0000 7f17c9756640 1 -- 172.21.15.19:0/2874978308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c4102a10 msgr2=0x7f17c4104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:26.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.995+0000 7f17c9756640 1 --2- 172.21.15.19:0/2874978308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c4102a10 0x7f17c4104e00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009900 tx=0x7f17bc02a410 comp rx=0 tx=0).stop 2024-04-25T12:04:26.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.995+0000 7f17c9756640 1 -- 172.21.15.19:0/2874978308 shutdown_connections 2024-04-25T12:04:26.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.995+0000 7f17c9756640 1 --2- 172.21.15.19:0/2874978308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4107c70 0x7f17c410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:26.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.995+0000 7f17c9756640 1 --2- 172.21.15.19:0/2874978308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c4105340 0x7f17c4107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:26.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.995+0000 7f17c9756640 1 --2- 172.21.15.19:0/2874978308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c4102a10 0x7f17c4104e00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:26.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.995+0000 7f17c9756640 1 -- 172.21.15.19:0/2874978308 >> 172.21.15.19:0/2874978308 conn(0x7f17c40fc400 msgr2=0x7f17c40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:26.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.995+0000 7f17c9756640 1 -- 172.21.15.19:0/2874978308 shutdown_connections 2024-04-25T12:04:26.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.995+0000 7f17c9756640 1 -- 172.21.15.19:0/2874978308 wait complete. 2024-04-25T12:04:26.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c9756640 1 Processor -- start 2024-04-25T12:04:26.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c9756640 1 -- start start 2024-04-25T12:04:26.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c9756640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c4102a10 0x7f17c4102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:26.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c9756640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4105340 0x7f17c41009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:26.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c9756640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c4107c70 0x7f17c4100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:26.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c9756640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17c410dc10 con 0x7f17c4102a10 2024-04-25T12:04:26.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c9756640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f17c410da90 con 0x7f17c4105340 2024-04-25T12:04:26.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c9756640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f17c410dd90 con 0x7f17c4107c70 2024-04-25T12:04:26.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c4102a10 0x7f17c4102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:26.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c4102a10 0x7f17c4102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39608/0 (socket says 172.21.15.19:39608) 2024-04-25T12:04:26.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c2ffd640 1 -- 172.21.15.19:0/304077172 learned_addr learned my addr 172.21.15.19:0/304077172 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:26.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c37fe640 1 --2- 172.21.15.19:0/304077172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c4107c70 0x7f17c4100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:26.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c27fc640 1 --2- 172.21.15.19:0/304077172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4105340 0x7f17c41009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:26.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c2ffd640 1 -- 172.21.15.19:0/304077172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c4107c70 msgr2=0x7f17c4100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:26.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c2ffd640 1 --2- 172.21.15.19:0/304077172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c4107c70 0x7f17c4100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:26.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.996+0000 7f17c2ffd640 1 -- 172.21.15.19:0/304077172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4105340 msgr2=0x7f17c41009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.997+0000 7f17c2ffd640 1 --2- 172.21.15.19:0/304077172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4105340 0x7f17c41009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.997+0000 7f17c2ffd640 1 -- 172.21.15.19:0/304077172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c4101790 con 0x7f17c4102a10 2024-04-25T12:04:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.997+0000 7f17c2ffd640 1 --2- 172.21.15.19:0/304077172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c4102a10 0x7f17c4102380 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f17bc005940 tx=0x7f17bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.997+0000 7f17bbfff640 1 -- 172.21.15.19:0/304077172 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc03f030 con 0x7f17c4102a10 2024-04-25T12:04:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.997+0000 7f17c9756640 1 -- 172.21.15.19:0/304077172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c41a23a0 con 0x7f17c4102a10 2024-04-25T12:04:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.997+0000 7f17bbfff640 1 -- 172.21.15.19:0/304077172 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f17bc003cd0 con 0x7f17c4102a10 2024-04-25T12:04:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.997+0000 7f17bbfff640 1 -- 172.21.15.19:0/304077172 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc035070 con 0x7f17c4102a10 2024-04-25T12:04:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.997+0000 7f17c9756640 1 -- 172.21.15.19:0/304077172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c41a28f0 con 0x7f17c4102a10 2024-04-25T12:04:26.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.998+0000 7f17c9756640 1 -- 172.21.15.19:0/304077172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c40fdef0 con 0x7f17c4102a10 2024-04-25T12:04:26.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:26.999+0000 7f17bbfff640 1 -- 172.21.15.19:0/304077172 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f17bc042030 con 0x7f17c4102a10 2024-04-25T12:04:26.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.000+0000 7f17bbfff640 1 --2- 172.21.15.19:0/304077172 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f17b00a3bb0 0x7f17b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:26.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.000+0000 7f17c27fc640 1 --2- 172.21.15.19:0/304077172 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f17b00a3bb0 0x7f17b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:26.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.000+0000 7f17bbfff640 1 -- 172.21.15.19:0/304077172 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f17bc0f0430 con 0x7f17c4102a10 2024-04-25T12:04:26.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.000+0000 7f17c27fc640 1 --2- 172.21.15.19:0/304077172 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f17b00a3bb0 0x7f17b00a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f17c41020f0 tx=0x7f17ac00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:27.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.004+0000 7f17bbfff640 1 -- 172.21.15.19:0/304077172 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17bc0bcc60 con 0x7f17c4102a10 2024-04-25T12:04:27.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.237+0000 7f17c9756640 1 -- 172.21.15.19:0/304077172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17c402d5c0 con 0x7f17c4102a10 2024-04-25T12:04:27.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.238+0000 7f17bbfff640 1 -- 172.21.15.19:0/304077172 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f17bc0c0f60 con 0x7f17c4102a10 2024-04-25T12:04:27.236 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:27.236 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T12:04:27.236 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 2 2024-04-25T12:04:27.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.240+0000 7f17c9756640 1 -- 172.21.15.19:0/304077172 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f17b00a3bb0 msgr2=0x7f17b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:27.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.240+0000 7f17c9756640 1 --2- 172.21.15.19:0/304077172 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f17b00a3bb0 0x7f17b00a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f17c41020f0 tx=0x7f17ac00b390 comp rx=0 tx=0).stop 2024-04-25T12:04:27.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.240+0000 7f17c9756640 1 -- 172.21.15.19:0/304077172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c4102a10 msgr2=0x7f17c4102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:27.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.240+0000 7f17c9756640 1 --2- 172.21.15.19:0/304077172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c4102a10 0x7f17c4102380 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f17bc005940 tx=0x7f17bc0042f0 comp rx=0 tx=0).stop 2024-04-25T12:04:27.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.240+0000 7f17c9756640 1 -- 172.21.15.19:0/304077172 shutdown_connections 2024-04-25T12:04:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.240+0000 7f17c9756640 1 --2- 172.21.15.19:0/304077172 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f17b00a3bb0 0x7f17b00a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.240+0000 7f17c9756640 1 --2- 172.21.15.19:0/304077172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17c4107c70 0x7f17c4100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.240+0000 7f17c9756640 1 --2- 172.21.15.19:0/304077172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17c4105340 0x7f17c41009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.240+0000 7f17c9756640 1 --2- 172.21.15.19:0/304077172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17c4102a10 0x7f17c4102380 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.240+0000 7f17c9756640 1 -- 172.21.15.19:0/304077172 >> 172.21.15.19:0/304077172 conn(0x7f17c40fc400 msgr2=0x7f17c4103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.240+0000 7f17c9756640 1 -- 172.21.15.19:0/304077172 shutdown_connections 2024-04-25T12:04:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:27.240+0000 7f17c9756640 1 -- 172.21.15.19:0/304077172 wait complete. 2024-04-25T12:04:27.845 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:27 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/304077172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:27.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:27 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/304077172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:28.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:27 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/304077172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:28.163 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:04:28 smithi150 systemd[1]: Starting Ceph mds.k for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:04:28.544 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:04:28 smithi150 podman[61875]: 2024-04-25 12:04:28.154883438 +0000 UTC m=+0.017928177 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:04:28.544 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:04:28 smithi150 podman[61875]: 2024-04-25 12:04:28.278309947 +0000 UTC m=+0.141354678 container create 0cf449dd81c5243949fea98c44bd5f9e04487d522f9f91715d4396e5e21d07e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T12:04:28.545 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:04:28 smithi150 podman[61875]: 2024-04-25 12:04:28.547785601 +0000 UTC m=+0.410830333 container init 0cf449dd81c5243949fea98c44bd5f9e04487d522f9f91715d4396e5e21d07e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T12:04:28.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:28 smithi169 ceph-mon[32710]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:28.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:28 smithi150 ceph-mon[32250]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:28.912 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:04:28 smithi150 podman[61875]: 2024-04-25 12:04:28.550479739 +0000 UTC m=+0.413524481 container start 0cf449dd81c5243949fea98c44bd5f9e04487d522f9f91715d4396e5e21d07e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T12:04:28.913 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:04:28 smithi150 ceph-mds[61894]: -- 172.21.15.150:0/4132407502 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d61a1da40 con 0x562d619f0c00 2024-04-25T12:04:28.913 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:04:28 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-k[61890]: starting mds.k at 2024-04-25T12:04:28.913 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:04:28 smithi150 bash[61875]: 0cf449dd81c5243949fea98c44bd5f9e04487d522f9f91715d4396e5e21d07e9 2024-04-25T12:04:28.913 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:04:28 smithi150 systemd[1]: Started Ceph mds.k for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T12:04:29.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:28 smithi019 ceph-mon[29213]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:29.249 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:29.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.358+0000 7f5612264640 1 -- 172.21.15.19:0/3399245836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f560c102a90 msgr2=0x7f560c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:29.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.358+0000 7f5612264640 1 --2- 172.21.15.19:0/3399245836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f560c102a90 0x7f560c104e80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009900 tx=0x7f55fc02a410 comp rx=0 tx=0).stop 2024-04-25T12:04:29.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.358+0000 7f5612264640 1 -- 172.21.15.19:0/3399245836 shutdown_connections 2024-04-25T12:04:29.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.358+0000 7f5612264640 1 --2- 172.21.15.19:0/3399245836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f560c107cf0 0x7f560c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:29.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.358+0000 7f5612264640 1 --2- 172.21.15.19:0/3399245836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c1053c0 0x7f560c1077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:29.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.358+0000 7f5612264640 1 --2- 172.21.15.19:0/3399245836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f560c102a90 0x7f560c104e80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:29.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.358+0000 7f5612264640 1 -- 172.21.15.19:0/3399245836 >> 172.21.15.19:0/3399245836 conn(0x7f560c0fc440 msgr2=0x7f560c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:29.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.358+0000 7f5612264640 1 -- 172.21.15.19:0/3399245836 shutdown_connections 2024-04-25T12:04:29.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.358+0000 7f5612264640 1 -- 172.21.15.19:0/3399245836 wait complete. 2024-04-25T12:04:29.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f5612264640 1 Processor -- start 2024-04-25T12:04:29.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f5612264640 1 -- start start 2024-04-25T12:04:29.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f5612264640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f560c102a90 0x7f560c102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:29.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f5612264640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c1053c0 0x7f560c100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:29.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f5612264640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f560c107cf0 0x7f560c100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:29.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f5612264640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f560c10dc30 con 0x7f560c102a90 2024-04-25T12:04:29.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f5612264640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f560c10dab0 con 0x7f560c1053c0 2024-04-25T12:04:29.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f5612264640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f560c10ddb0 con 0x7f560c107cf0 2024-04-25T12:04:29.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f560bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f560c102a90 0x7f560c102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:29.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f560bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f560c102a90 0x7f560c102400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39634/0 (socket says 172.21.15.19:39634) 2024-04-25T12:04:29.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f560bfff640 1 -- 172.21.15.19:0/232967168 learned_addr learned my addr 172.21.15.19:0/232967168 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:29.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f5610801640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f560c107cf0 0x7f560c100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:29.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f560b7fe640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c1053c0 0x7f560c100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:29.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f560bfff640 1 -- 172.21.15.19:0/232967168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f560c107cf0 msgr2=0x7f560c100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:29.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f560bfff640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f560c107cf0 0x7f560c100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:29.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f560bfff640 1 -- 172.21.15.19:0/232967168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c1053c0 msgr2=0x7f560c100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:29.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f560bfff640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c1053c0 0x7f560c100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:29.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f560bfff640 1 -- 172.21.15.19:0/232967168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c101810 con 0x7f560c102a90 2024-04-25T12:04:29.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.359+0000 7f560b7fe640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c1053c0 0x7f560c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:04:29.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.360+0000 7f560bfff640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f560c102a90 0x7f560c102400 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f55fc005940 tx=0x7f55fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:29.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.360+0000 7f56097fa640 1 -- 172.21.15.19:0/232967168 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc03f030 con 0x7f560c102a90 2024-04-25T12:04:29.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.360+0000 7f56097fa640 1 -- 172.21.15.19:0/232967168 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f55fc003cd0 con 0x7f560c102a90 2024-04-25T12:04:29.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.360+0000 7f5612264640 1 -- 172.21.15.19:0/232967168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560c1a23c0 con 0x7f560c102a90 2024-04-25T12:04:29.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.360+0000 7f56097fa640 1 -- 172.21.15.19:0/232967168 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc035070 con 0x7f560c102a90 2024-04-25T12:04:29.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.360+0000 7f5612264640 1 -- 172.21.15.19:0/232967168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560c1a2960 con 0x7f560c102a90 2024-04-25T12:04:29.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.361+0000 7f5612264640 1 -- 172.21.15.19:0/232967168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560c0fdf30 con 0x7f560c102a90 2024-04-25T12:04:29.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.362+0000 7f56097fa640 1 -- 172.21.15.19:0/232967168 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f55fc042030 con 0x7f560c102a90 2024-04-25T12:04:29.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.363+0000 7f56097fa640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f55f40a3c30 0x7f55f40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:29.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.363+0000 7f560b7fe640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f55f40a3c30 0x7f55f40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:29.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.363+0000 7f56097fa640 1 -- 172.21.15.19:0/232967168 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f55fc0f0530 con 0x7f560c102a90 2024-04-25T12:04:29.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.363+0000 7f560b7fe640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f55f40a3c30 0x7f55f40a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f560c102170 tx=0x7f5600005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:29.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.368+0000 7f56097fa640 1 -- 172.21.15.19:0/232967168 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55fc0bd3b0 con 0x7f560c102a90 2024-04-25T12:04:29.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.597+0000 7f5612264640 1 -- 172.21.15.19:0/232967168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f560c02d5c0 con 0x7f560c102a90 2024-04-25T12:04:29.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.597+0000 7f56097fa640 1 -- 172.21.15.19:0/232967168 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f55fc0bccb0 con 0x7f560c102a90 2024-04-25T12:04:29.595 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:29.596 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T12:04:29.596 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 3 2024-04-25T12:04:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.599+0000 7f5612264640 1 -- 172.21.15.19:0/232967168 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f55f40a3c30 msgr2=0x7f55f40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.599+0000 7f5612264640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f55f40a3c30 0x7f55f40a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f560c102170 tx=0x7f5600005f90 comp rx=0 tx=0).stop 2024-04-25T12:04:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.599+0000 7f5612264640 1 -- 172.21.15.19:0/232967168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f560c102a90 msgr2=0x7f560c102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.599+0000 7f5612264640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f560c102a90 0x7f560c102400 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f55fc005940 tx=0x7f55fc0042f0 comp rx=0 tx=0).stop 2024-04-25T12:04:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.599+0000 7f5612264640 1 -- 172.21.15.19:0/232967168 shutdown_connections 2024-04-25T12:04:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.599+0000 7f5612264640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f55f40a3c30 0x7f55f40a60f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.599+0000 7f5612264640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f560c107cf0 0x7f560c100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.599+0000 7f5612264640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c1053c0 0x7f560c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.599+0000 7f5612264640 1 --2- 172.21.15.19:0/232967168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f560c102a90 0x7f560c102400 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.599+0000 7f5612264640 1 -- 172.21.15.19:0/232967168 >> 172.21.15.19:0/232967168 conn(0x7f560c0fc440 msgr2=0x7f560c1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.599+0000 7f5612264640 1 -- 172.21.15.19:0/232967168 shutdown_connections 2024-04-25T12:04:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:29.599+0000 7f5612264640 1 -- 172.21.15.19:0/232967168 wait complete. 2024-04-25T12:04:29.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:29 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.150:6832/4176941259,v1:172.21.15.150:6833/4176941259] up:boot 2024-04-25T12:04:29.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:29 smithi169 ceph-mon[32710]: fsmap 2 up:standby 2024-04-25T12:04:29.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T12:04:29.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:29.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:29.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:29.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:29.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:29.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:29 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:29.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:29 smithi169 ceph-mon[32710]: Deploying daemon mds.j on smithi019 2024-04-25T12:04:29.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:29 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/232967168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:29.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:29 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.150:6832/4176941259,v1:172.21.15.150:6833/4176941259] up:boot 2024-04-25T12:04:29.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:29 smithi150 ceph-mon[32250]: fsmap 2 up:standby 2024-04-25T12:04:29.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T12:04:29.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:29.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:29.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:29.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:29.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:29.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:29 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:29.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:29 smithi150 ceph-mon[32250]: Deploying daemon mds.j on smithi019 2024-04-25T12:04:29.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:29 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/232967168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:30.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:29 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.150:6832/4176941259,v1:172.21.15.150:6833/4176941259] up:boot 2024-04-25T12:04:30.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:29 smithi019 ceph-mon[29213]: fsmap 2 up:standby 2024-04-25T12:04:30.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T12:04:30.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:30.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:30.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:30.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:30.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:30.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:29 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:30.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:29 smithi019 ceph-mon[29213]: Deploying daemon mds.j on smithi019 2024-04-25T12:04:30.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:29 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/232967168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:30.973 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:30 smithi150 ceph-mon[32250]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:31.015 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:30 smithi019 ceph-mon[29213]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:31.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:30 smithi169 ceph-mon[32710]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:31.304 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:04:31 smithi019 podman[70374]: 2024-04-25 12:04:31.258088825 +0000 UTC m=+0.134528684 container create 66947b32c094f22f69abc2e938048312a0eebe7ca9cc2bf4934348512722fdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T12:04:31.609 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:31.634 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:04:31 smithi019 podman[70374]: 2024-04-25 12:04:31.485773682 +0000 UTC m=+0.362213540 container init 66947b32c094f22f69abc2e938048312a0eebe7ca9cc2bf4934348512722fdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T12:04:31.634 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:04:31 smithi019 podman[70374]: 2024-04-25 12:04:31.488907961 +0000 UTC m=+0.365347827 container start 66947b32c094f22f69abc2e938048312a0eebe7ca9cc2bf4934348512722fdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-25T12:04:31.635 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:04:31 smithi019 ceph-mds[70393]: -- 172.21.15.19:0/1289953836 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557774d2da40 con 0x557774d00800 2024-04-25T12:04:31.635 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:04:31 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-j[70389]: starting mds.j at 2024-04-25T12:04:31.635 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:04:31 smithi019 bash[70374]: 66947b32c094f22f69abc2e938048312a0eebe7ca9cc2bf4934348512722fdc4 2024-04-25T12:04:31.635 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:04:31 smithi019 systemd[1]: Started Ceph mds.j for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T12:04:31.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.722+0000 7f9f90635640 1 -- 172.21.15.19:0/1528297056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f88102a10 msgr2=0x7f9f88104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:31.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.722+0000 7f9f90635640 1 --2- 172.21.15.19:0/1528297056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f88102a10 0x7f9f88104e00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009900 tx=0x7f9f8002a410 comp rx=0 tx=0).stop 2024-04-25T12:04:31.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.722+0000 7f9f90635640 1 -- 172.21.15.19:0/1528297056 shutdown_connections 2024-04-25T12:04:31.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.722+0000 7f9f90635640 1 --2- 172.21.15.19:0/1528297056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f88107c70 0x7f9f8810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:31.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.722+0000 7f9f90635640 1 --2- 172.21.15.19:0/1528297056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f88105340 0x7f9f88107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:31.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.722+0000 7f9f90635640 1 --2- 172.21.15.19:0/1528297056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f88102a10 0x7f9f88104e00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:31.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.722+0000 7f9f90635640 1 -- 172.21.15.19:0/1528297056 >> 172.21.15.19:0/1528297056 conn(0x7f9f880fc400 msgr2=0x7f9f880fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:31.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.722+0000 7f9f90635640 1 -- 172.21.15.19:0/1528297056 shutdown_connections 2024-04-25T12:04:31.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.722+0000 7f9f90635640 1 -- 172.21.15.19:0/1528297056 wait complete. 2024-04-25T12:04:31.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f90635640 1 Processor -- start 2024-04-25T12:04:31.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f90635640 1 -- start start 2024-04-25T12:04:31.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f90635640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f88102a10 0x7f9f88102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:31.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f90635640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f88105340 0x7f9f881009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:31.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f90635640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f88107c70 0x7f9f88100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:31.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f90635640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8810dc20 con 0x7f9f88107c70 2024-04-25T12:04:31.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f90635640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8810daa0 con 0x7f9f88105340 2024-04-25T12:04:31.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f90635640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8810dda0 con 0x7f9f88102a10 2024-04-25T12:04:31.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f8ebd2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f88107c70 0x7f9f88100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:31.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f8ebd2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f88107c70 0x7f9f88100f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53002/0 (socket says 172.21.15.19:53002) 2024-04-25T12:04:31.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f8ebd2640 1 -- 172.21.15.19:0/256331134 learned_addr learned my addr 172.21.15.19:0/256331134 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:31.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f8dbd0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f88105340 0x7f9f881009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:31.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f8e3d1640 1 --2- 172.21.15.19:0/256331134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f88102a10 0x7f9f88102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:31.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f8ebd2640 1 -- 172.21.15.19:0/256331134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f88102a10 msgr2=0x7f9f88102380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:31.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f8ebd2640 1 --2- 172.21.15.19:0/256331134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f88102a10 0x7f9f88102380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:31.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f8ebd2640 1 -- 172.21.15.19:0/256331134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f88105340 msgr2=0x7f9f881009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:31.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.723+0000 7f9f8ebd2640 1 --2- 172.21.15.19:0/256331134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f88105340 0x7f9f881009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:31.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.724+0000 7f9f8ebd2640 1 -- 172.21.15.19:0/256331134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f88101790 con 0x7f9f88107c70 2024-04-25T12:04:31.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.724+0000 7f9f8ebd2640 1 --2- 172.21.15.19:0/256331134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f88107c70 0x7f9f88100f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800eaa0 tx=0x7f9f7800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:31.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.724+0000 7f9f877fe640 1 -- 172.21.15.19:0/256331134 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7800cdb0 con 0x7f9f88107c70 2024-04-25T12:04:31.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.724+0000 7f9f877fe640 1 -- 172.21.15.19:0/256331134 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9f78005020 con 0x7f9f88107c70 2024-04-25T12:04:31.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.724+0000 7f9f877fe640 1 -- 172.21.15.19:0/256331134 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f780116d0 con 0x7f9f88107c70 2024-04-25T12:04:31.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.724+0000 7f9f90635640 1 -- 172.21.15.19:0/256331134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f881a23c0 con 0x7f9f88107c70 2024-04-25T12:04:31.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.724+0000 7f9f90635640 1 -- 172.21.15.19:0/256331134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f881a2960 con 0x7f9f88107c70 2024-04-25T12:04:31.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.725+0000 7f9f90635640 1 -- 172.21.15.19:0/256331134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f880fdef0 con 0x7f9f88107c70 2024-04-25T12:04:31.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.726+0000 7f9f877fe640 1 -- 172.21.15.19:0/256331134 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9f780051d0 con 0x7f9f88107c70 2024-04-25T12:04:31.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.727+0000 7f9f877fe640 1 --2- 172.21.15.19:0/256331134 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9f700a3c30 0x7f9f700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:31.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.727+0000 7f9f8e3d1640 1 --2- 172.21.15.19:0/256331134 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9f700a3c30 0x7f9f700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:31.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.727+0000 7f9f877fe640 1 -- 172.21.15.19:0/256331134 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9f780d0150 con 0x7f9f88107c70 2024-04-25T12:04:31.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.728+0000 7f9f8e3d1640 1 --2- 172.21.15.19:0/256331134 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9f700a3c30 0x7f9f700a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f80000c00 tx=0x7f9f80031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:31.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.732+0000 7f9f877fe640 1 -- 172.21.15.19:0/256331134 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f78016030 con 0x7f9f88107c70 2024-04-25T12:04:31.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.961+0000 7f9f90635640 1 -- 172.21.15.19:0/256331134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f88062b50 con 0x7f9f88107c70 2024-04-25T12:04:31.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.961+0000 7f9f877fe640 1 -- 172.21.15.19:0/256331134 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f9f7809cfd0 con 0x7f9f88107c70 2024-04-25T12:04:31.960 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:31.960 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T12:04:31.960 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 4 2024-04-25T12:04:31.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.963+0000 7f9f90635640 1 -- 172.21.15.19:0/256331134 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9f700a3c30 msgr2=0x7f9f700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.963+0000 7f9f90635640 1 --2- 172.21.15.19:0/256331134 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9f700a3c30 0x7f9f700a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f80000c00 tx=0x7f9f80031510 comp rx=0 tx=0).stop 2024-04-25T12:04:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.963+0000 7f9f90635640 1 -- 172.21.15.19:0/256331134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f88107c70 msgr2=0x7f9f88100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.963+0000 7f9f90635640 1 --2- 172.21.15.19:0/256331134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f88107c70 0x7f9f88100f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800eaa0 tx=0x7f9f7800ef60 comp rx=0 tx=0).stop 2024-04-25T12:04:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.963+0000 7f9f90635640 1 -- 172.21.15.19:0/256331134 shutdown_connections 2024-04-25T12:04:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.963+0000 7f9f90635640 1 --2- 172.21.15.19:0/256331134 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9f700a3c30 0x7f9f700a60f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.963+0000 7f9f90635640 1 --2- 172.21.15.19:0/256331134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f88107c70 0x7f9f88100f00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.963+0000 7f9f90635640 1 --2- 172.21.15.19:0/256331134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f88105340 0x7f9f881009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.963+0000 7f9f90635640 1 --2- 172.21.15.19:0/256331134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f88102a10 0x7f9f88102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.963+0000 7f9f90635640 1 -- 172.21.15.19:0/256331134 >> 172.21.15.19:0/256331134 conn(0x7f9f880fc400 msgr2=0x7f9f88103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.963+0000 7f9f90635640 1 -- 172.21.15.19:0/256331134 shutdown_connections 2024-04-25T12:04:31.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:31.963+0000 7f9f90635640 1 -- 172.21.15.19:0/256331134 wait complete. 2024-04-25T12:04:32.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:32 smithi169 ceph-mon[32710]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:32.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:32 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:32.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:32 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:32.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:32 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:32.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:32 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:32.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:32 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:32.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:32 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:32.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:32 smithi169 ceph-mon[32710]: Deploying daemon mds.d on smithi019 2024-04-25T12:04:32.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:32 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.19:6834/3292992712,v1:172.21.15.19:6835/3292992712] up:boot 2024-04-25T12:04:32.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:32 smithi169 ceph-mon[32710]: fsmap 3 up:standby 2024-04-25T12:04:32.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:32 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T12:04:32.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:32 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/256331134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:32.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:32 smithi150 ceph-mon[32250]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:32.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:32 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:32.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:32 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:32.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:32 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:32.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:32 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:32.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:32 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:32.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:32 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:32.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:32 smithi150 ceph-mon[32250]: Deploying daemon mds.d on smithi019 2024-04-25T12:04:32.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:32 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.19:6834/3292992712,v1:172.21.15.19:6835/3292992712] up:boot 2024-04-25T12:04:32.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:32 smithi150 ceph-mon[32250]: fsmap 3 up:standby 2024-04-25T12:04:32.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:32 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T12:04:32.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:32 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/256331134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:33.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:32 smithi019 ceph-mon[29213]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:33.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:32 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:33.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:32 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:33.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:32 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:33.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:32 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:33.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:32 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:33.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:32 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:33.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:32 smithi019 ceph-mon[29213]: Deploying daemon mds.d on smithi019 2024-04-25T12:04:33.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:32 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.19:6834/3292992712,v1:172.21.15.19:6835/3292992712] up:boot 2024-04-25T12:04:33.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:32 smithi019 ceph-mon[29213]: fsmap 3 up:standby 2024-04-25T12:04:33.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:32 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T12:04:33.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:32 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/256331134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:33.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:34.020 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:04:34 smithi019 systemd[1]: Starting Ceph mds.d for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:04:34.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.091+0000 7fcccffb4640 1 -- 172.21.15.19:0/4137337819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc81009c0 msgr2=0x7fccc8100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:34.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.091+0000 7fcccffb4640 1 --2- 172.21.15.19:0/4137337819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc81009c0 0x7fccc8100e70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009ca0 tx=0x7fccc002a8c0 comp rx=0 tx=0).stop 2024-04-25T12:04:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.091+0000 7fcccffb4640 1 -- 172.21.15.19:0/4137337819 shutdown_connections 2024-04-25T12:04:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.091+0000 7fcccffb4640 1 --2- 172.21.15.19:0/4137337819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc81009c0 0x7fccc8100e70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.091+0000 7fcccffb4640 1 --2- 172.21.15.19:0/4137337819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccc8100000 0x7fccc8100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.091+0000 7fcccffb4640 1 --2- 172.21.15.19:0/4137337819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccc8102510 0x7fccc8102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:34.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.091+0000 7fcccffb4640 1 -- 172.21.15.19:0/4137337819 >> 172.21.15.19:0/4137337819 conn(0x7fccc80fbb80 msgr2=0x7fccc80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.091+0000 7fcccffb4640 1 -- 172.21.15.19:0/4137337819 shutdown_connections 2024-04-25T12:04:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.091+0000 7fcccffb4640 1 -- 172.21.15.19:0/4137337819 wait complete. 2024-04-25T12:04:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.091+0000 7fcccffb4640 1 Processor -- start 2024-04-25T12:04:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.091+0000 7fcccffb4640 1 -- start start 2024-04-25T12:04:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fcccffb4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccc8100000 0x7fccc819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:34.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fcccffb4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccc81009c0 0x7fccc819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:34.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fcccffb4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc8102510 0x7fccc81a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:34.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fcccffb4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fccc810fe00 con 0x7fccc8102510 2024-04-25T12:04:34.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fcccffb4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fccc810fc80 con 0x7fccc81009c0 2024-04-25T12:04:34.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fcccffb4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fccc810ff80 con 0x7fccc8100000 2024-04-25T12:04:34.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fccce551640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc8102510 0x7fccc81a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:34.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fccce551640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc8102510 0x7fccc81a1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53014/0 (socket says 172.21.15.19:53014) 2024-04-25T12:04:34.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fccce551640 1 -- 172.21.15.19:0/2623278265 learned_addr learned my addr 172.21.15.19:0/2623278265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:34.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fcccdd50640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccc8100000 0x7fccc819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:34.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fcccd54f640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccc81009c0 0x7fccc819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:34.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fccce551640 1 -- 172.21.15.19:0/2623278265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccc8100000 msgr2=0x7fccc819a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:34.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fccce551640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccc8100000 0x7fccc819a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:34.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fccce551640 1 -- 172.21.15.19:0/2623278265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccc81009c0 msgr2=0x7fccc819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:34.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fccce551640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccc81009c0 0x7fccc819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:34.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fccce551640 1 -- 172.21.15.19:0/2623278265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc81a1fd0 con 0x7fccc8102510 2024-04-25T12:04:34.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fcccd54f640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccc81009c0 0x7fccc819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:04:34.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fccce551640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc8102510 0x7fccc81a1960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009dc0 tx=0x7fccc002f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:34.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fccbeffd640 1 -- 172.21.15.19:0/2623278265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0003cf0 con 0x7fccc8102510 2024-04-25T12:04:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fccbeffd640 1 -- 172.21.15.19:0/2623278265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fccc0004510 con 0x7fccc8102510 2024-04-25T12:04:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fcccffb4640 1 -- 172.21.15.19:0/2623278265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc81a2260 con 0x7fccc8102510 2024-04-25T12:04:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.092+0000 7fccbeffd640 1 -- 172.21.15.19:0/2623278265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc002fdb0 con 0x7fccc8102510 2024-04-25T12:04:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.093+0000 7fcccffb4640 1 -- 172.21.15.19:0/2623278265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc81a27b0 con 0x7fccc8102510 2024-04-25T12:04:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.094+0000 7fcccffb4640 1 -- 172.21.15.19:0/2623278265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc810bf30 con 0x7fccc8102510 2024-04-25T12:04:34.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.095+0000 7fccbeffd640 1 -- 172.21.15.19:0/2623278265 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fccc00046c0 con 0x7fccc8102510 2024-04-25T12:04:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.095+0000 7fccbeffd640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fccb40a3bb0 0x7fccb40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.096+0000 7fcccdd50640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fccb40a3bb0 0x7fccb40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.096+0000 7fccbeffd640 1 -- 172.21.15.19:0/2623278265 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fccc00f1760 con 0x7fccc8102510 2024-04-25T12:04:34.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.096+0000 7fcccdd50640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fccb40a3bb0 0x7fccb40a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fccb8005fd0 tx=0x7fccb8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:34.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.101+0000 7fccbeffd640 1 -- 172.21.15.19:0/2623278265 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fccc00be5e0 con 0x7fccc8102510 2024-04-25T12:04:34.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.333+0000 7fcccffb4640 1 -- 172.21.15.19:0/2623278265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccc802d5c0 con 0x7fccc8102510 2024-04-25T12:04:34.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.333+0000 7fccbeffd640 1 -- 172.21.15.19:0/2623278265 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7fccc00bdee0 con 0x7fccc8102510 2024-04-25T12:04:34.331 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:34.332 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T12:04:34.332 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 4 2024-04-25T12:04:34.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.335+0000 7fcccffb4640 1 -- 172.21.15.19:0/2623278265 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fccb40a3bb0 msgr2=0x7fccb40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:34.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.335+0000 7fcccffb4640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fccb40a3bb0 0x7fccb40a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fccb8005fd0 tx=0x7fccb8005850 comp rx=0 tx=0).stop 2024-04-25T12:04:34.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.335+0000 7fcccffb4640 1 -- 172.21.15.19:0/2623278265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc8102510 msgr2=0x7fccc81a1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:34.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.335+0000 7fcccffb4640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc8102510 0x7fccc81a1960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009dc0 tx=0x7fccc002f460 comp rx=0 tx=0).stop 2024-04-25T12:04:34.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.335+0000 7fcccffb4640 1 -- 172.21.15.19:0/2623278265 shutdown_connections 2024-04-25T12:04:34.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.335+0000 7fcccffb4640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fccb40a3bb0 0x7fccb40a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:34.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.335+0000 7fcccffb4640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fccc8102510 0x7fccc81a1960 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:34.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.335+0000 7fcccffb4640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccc81009c0 0x7fccc819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:34.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.335+0000 7fcccffb4640 1 --2- 172.21.15.19:0/2623278265 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccc8100000 0x7fccc819a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:34.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.335+0000 7fcccffb4640 1 -- 172.21.15.19:0/2623278265 >> 172.21.15.19:0/2623278265 conn(0x7fccc80fbb80 msgr2=0x7fccc80fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:34.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.335+0000 7fcccffb4640 1 -- 172.21.15.19:0/2623278265 shutdown_connections 2024-04-25T12:04:34.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:34.335+0000 7fcccffb4640 1 -- 172.21.15.19:0/2623278265 wait complete. 2024-04-25T12:04:34.513 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:04:34 smithi019 podman[70756]: 2024-04-25 12:04:34.152583932 +0000 UTC m=+0.019191525 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:04:34.513 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:04:34 smithi019 podman[70756]: 2024-04-25 12:04:34.272152596 +0000 UTC m=+0.138760175 container create 3d49cfc6549a0f82515144f7ecabf973d284ba0166fc9e8665809faa5a61f5f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T12:04:34.513 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:04:34 smithi019 podman[70756]: 2024-04-25 12:04:34.516447946 +0000 UTC m=+0.383055525 container init 3d49cfc6549a0f82515144f7ecabf973d284ba0166fc9e8665809faa5a61f5f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:04:34.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:34 smithi019 ceph-mon[29213]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:34.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:34 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2623278265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:34.798 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:04:34 smithi019 podman[70756]: 2024-04-25 12:04:34.519184884 +0000 UTC m=+0.385792471 container start 3d49cfc6549a0f82515144f7ecabf973d284ba0166fc9e8665809faa5a61f5f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T12:04:34.798 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:04:34 smithi019 ceph-mds[70777]: -- 172.21.15.19:0/3557075951 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56395d67fa40 con 0x56395d652800 2024-04-25T12:04:34.798 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:04:34 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-d[70773]: starting mds.d at 2024-04-25T12:04:34.798 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:04:34 smithi019 bash[70756]: 3d49cfc6549a0f82515144f7ecabf973d284ba0166fc9e8665809faa5a61f5f3 2024-04-25T12:04:34.798 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:04:34 smithi019 systemd[1]: Started Ceph mds.d for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T12:04:34.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:34 smithi169 ceph-mon[32710]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:34.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:34 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2623278265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:34.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:34 smithi150 ceph-mon[32250]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:34.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:34 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2623278265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:35.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:35 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.19:6836/3684868666,v1:172.21.15.19:6837/3684868666] up:boot 2024-04-25T12:04:35.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:35 smithi169 ceph-mon[32710]: fsmap 4 up:standby 2024-04-25T12:04:35.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:35 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T12:04:35.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:35 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:35.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:35 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:35.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:35 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:35.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:35 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:35.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:35 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:35.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:35 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:35.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:35 smithi169 ceph-mon[32710]: Deploying daemon mds.e on smithi150 2024-04-25T12:04:35.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:35 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.19:6836/3684868666,v1:172.21.15.19:6837/3684868666] up:boot 2024-04-25T12:04:35.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:35 smithi150 ceph-mon[32250]: fsmap 4 up:standby 2024-04-25T12:04:35.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:35 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T12:04:35.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:35 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:35.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:35 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:35.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:35 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:35.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:35 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:35.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:35 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:35.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:35 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:35.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:35 smithi150 ceph-mon[32250]: Deploying daemon mds.e on smithi150 2024-04-25T12:04:36.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:35 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.19:6836/3684868666,v1:172.21.15.19:6837/3684868666] up:boot 2024-04-25T12:04:36.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:35 smithi019 ceph-mon[29213]: fsmap 4 up:standby 2024-04-25T12:04:36.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T12:04:36.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:36.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:36.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:36.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:36.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:36.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:35 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:36.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:35 smithi019 ceph-mon[29213]: Deploying daemon mds.e on smithi150 2024-04-25T12:04:36.346 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.463+0000 7f7012315640 1 -- 172.21.15.19:0/542169018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c102a30 msgr2=0x7f700c104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.463+0000 7f7012315640 1 --2- 172.21.15.19:0/542169018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c102a30 0x7f700c104e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009900 tx=0x7f6ffc02a410 comp rx=0 tx=0).stop 2024-04-25T12:04:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.463+0000 7f7012315640 1 -- 172.21.15.19:0/542169018 shutdown_connections 2024-04-25T12:04:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.463+0000 7f7012315640 1 --2- 172.21.15.19:0/542169018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f700c107c90 0x7f700c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.463+0000 7f7012315640 1 --2- 172.21.15.19:0/542169018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f700c105360 0x7f700c107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.463+0000 7f7012315640 1 --2- 172.21.15.19:0/542169018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c102a30 0x7f700c104e20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.463+0000 7f7012315640 1 -- 172.21.15.19:0/542169018 >> 172.21.15.19:0/542169018 conn(0x7f700c0fc440 msgr2=0x7f700c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.463+0000 7f7012315640 1 -- 172.21.15.19:0/542169018 shutdown_connections 2024-04-25T12:04:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.463+0000 7f7012315640 1 -- 172.21.15.19:0/542169018 wait complete. 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.464+0000 7f7012315640 1 Processor -- start 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.464+0000 7f7012315640 1 -- start start 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.464+0000 7f7012315640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c102a30 0x7f700c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.464+0000 7f7012315640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f700c105360 0x7f700c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.464+0000 7f7012315640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f700c107c90 0x7f700c19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.464+0000 7f7012315640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f700c10dc30 con 0x7f700c102a30 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.464+0000 7f7012315640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f700c10dab0 con 0x7f700c107c90 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.464+0000 7f7012315640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f700c10ddb0 con 0x7f700c105360 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.464+0000 7f700bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c102a30 0x7f700c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f700bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c102a30 0x7f700c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53052/0 (socket says 172.21.15.19:53052) 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f700bfff640 1 -- 172.21.15.19:0/2568336629 learned_addr learned my addr 172.21.15.19:0/2568336629 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f70108b2640 1 --2- 172.21.15.19:0/2568336629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f700c107c90 0x7f700c19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f700b7fe640 1 --2- 172.21.15.19:0/2568336629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f700c105360 0x7f700c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f700bfff640 1 -- 172.21.15.19:0/2568336629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f700c105360 msgr2=0x7f700c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f700bfff640 1 --2- 172.21.15.19:0/2568336629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f700c105360 0x7f700c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:36.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f700bfff640 1 -- 172.21.15.19:0/2568336629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f700c107c90 msgr2=0x7f700c19f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:36.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f700bfff640 1 --2- 172.21.15.19:0/2568336629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f700c107c90 0x7f700c19f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:36.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f700bfff640 1 -- 172.21.15.19:0/2568336629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700c19ff20 con 0x7f700c102a30 2024-04-25T12:04:36.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f700bfff640 1 --2- 172.21.15.19:0/2568336629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c102a30 0x7f700c198300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc005940 tx=0x7f6ffc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:36.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f70097fa640 1 -- 172.21.15.19:0/2568336629 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc03f030 con 0x7f700c102a30 2024-04-25T12:04:36.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f7012315640 1 -- 172.21.15.19:0/2568336629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700c1a01b0 con 0x7f700c102a30 2024-04-25T12:04:36.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f70097fa640 1 -- 172.21.15.19:0/2568336629 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6ffc003cd0 con 0x7f700c102a30 2024-04-25T12:04:36.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f70097fa640 1 -- 172.21.15.19:0/2568336629 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc035070 con 0x7f700c102a30 2024-04-25T12:04:36.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.465+0000 7f7012315640 1 -- 172.21.15.19:0/2568336629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f700c1a06a0 con 0x7f700c102a30 2024-04-25T12:04:36.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.466+0000 7f7012315640 1 -- 172.21.15.19:0/2568336629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f700c0fdf30 con 0x7f700c102a30 2024-04-25T12:04:36.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.467+0000 7f70097fa640 1 -- 172.21.15.19:0/2568336629 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6ffc042030 con 0x7f700c102a30 2024-04-25T12:04:36.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.468+0000 7f70097fa640 1 --2- 172.21.15.19:0/2568336629 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6ff80a3bb0 0x7f6ff80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:36.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.468+0000 7f700b7fe640 1 --2- 172.21.15.19:0/2568336629 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6ff80a3bb0 0x7f6ff80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:36.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.468+0000 7f70097fa640 1 -- 172.21.15.19:0/2568336629 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6ffc0f06c0 con 0x7f700c102a30 2024-04-25T12:04:36.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.469+0000 7f700b7fe640 1 --2- 172.21.15.19:0/2568336629 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6ff80a3bb0 0x7f6ff80a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4005fd0 tx=0x7f6ff4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:36.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.473+0000 7f70097fa640 1 -- 172.21.15.19:0/2568336629 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ffc0bd4a0 con 0x7f700c102a30 2024-04-25T12:04:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.699+0000 7f7012315640 1 -- 172.21.15.19:0/2568336629 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f700c02d5c0 con 0x7f700c102a30 2024-04-25T12:04:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.700+0000 7f70097fa640 1 -- 172.21.15.19:0/2568336629 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7f6ffc0bcda0 con 0x7f700c102a30 2024-04-25T12:04:36.698 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:36.699 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T12:04:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 5 2024-04-25T12:04:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.702+0000 7f7012315640 1 -- 172.21.15.19:0/2568336629 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6ff80a3bb0 msgr2=0x7f6ff80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.702+0000 7f7012315640 1 --2- 172.21.15.19:0/2568336629 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6ff80a3bb0 0x7f6ff80a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4005fd0 tx=0x7f6ff4005850 comp rx=0 tx=0).stop 2024-04-25T12:04:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.702+0000 7f7012315640 1 -- 172.21.15.19:0/2568336629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c102a30 msgr2=0x7f700c198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.702+0000 7f7012315640 1 --2- 172.21.15.19:0/2568336629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c102a30 0x7f700c198300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc005940 tx=0x7f6ffc0043b0 comp rx=0 tx=0).stop 2024-04-25T12:04:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.702+0000 7f7012315640 1 -- 172.21.15.19:0/2568336629 shutdown_connections 2024-04-25T12:04:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.702+0000 7f7012315640 1 --2- 172.21.15.19:0/2568336629 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6ff80a3bb0 0x7f6ff80a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.702+0000 7f7012315640 1 --2- 172.21.15.19:0/2568336629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f700c107c90 0x7f700c19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.702+0000 7f7012315640 1 --2- 172.21.15.19:0/2568336629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f700c105360 0x7f700c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.702+0000 7f7012315640 1 --2- 172.21.15.19:0/2568336629 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f700c102a30 0x7f700c198300 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.702+0000 7f7012315640 1 -- 172.21.15.19:0/2568336629 >> 172.21.15.19:0/2568336629 conn(0x7f700c0fc440 msgr2=0x7f700c103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.702+0000 7f7012315640 1 -- 172.21.15.19:0/2568336629 shutdown_connections 2024-04-25T12:04:36.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:36.702+0000 7f7012315640 1 -- 172.21.15.19:0/2568336629 wait complete. 2024-04-25T12:04:36.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:36 smithi150 ceph-mon[32250]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:36.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:36 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2568336629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:37.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:36 smithi019 ceph-mon[29213]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:37.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:36 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2568336629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:37.082 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:36 smithi169 ceph-mon[32710]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:37.083 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:36 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2568336629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:37.359 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:04:37 smithi150 systemd[1]: Starting Ceph mds.e for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:04:37.663 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:04:37 smithi150 podman[62194]: 2024-04-25 12:04:37.264654546 +0000 UTC m=+0.017833848 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:04:37.663 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:04:37 smithi150 podman[62194]: 2024-04-25 12:04:37.426634495 +0000 UTC m=+0.179813808 container create 58a4be0bae3252b3ea38cc15c60102ee78652367a3f0444bf815b6947ccd2225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T12:04:38.163 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:04:37 smithi150 podman[62194]: 2024-04-25 12:04:37.687694719 +0000 UTC m=+0.440873998 container init 58a4be0bae3252b3ea38cc15c60102ee78652367a3f0444bf815b6947ccd2225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T12:04:38.163 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:04:37 smithi150 podman[62194]: 2024-04-25 12:04:37.690329211 +0000 UTC m=+0.443508490 container start 58a4be0bae3252b3ea38cc15c60102ee78652367a3f0444bf815b6947ccd2225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T12:04:38.163 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:04:37 smithi150 ceph-mds[62215]: -- 172.21.15.150:0/875597020 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff70f5fa40 con 0x55ff70f33000 2024-04-25T12:04:38.163 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:04:37 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-e[62211]: starting mds.e at 2024-04-25T12:04:38.163 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:04:37 smithi150 bash[62194]: 58a4be0bae3252b3ea38cc15c60102ee78652367a3f0444bf815b6947ccd2225 2024-04-25T12:04:38.163 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:04:37 smithi150 systemd[1]: Started Ceph mds.e for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T12:04:38.712 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.829+0000 7f925d39b640 1 -- 172.21.15.19:0/2775280969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9258103b70 msgr2=0x7f925810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.829+0000 7f925d39b640 1 --2- 172.21.15.19:0/2775280969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9258103b70 0x7f925810a400 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9250009ca0 tx=0x7f925002a8c0 comp rx=0 tx=0).stop 2024-04-25T12:04:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.830+0000 7f925d39b640 1 -- 172.21.15.19:0/2775280969 shutdown_connections 2024-04-25T12:04:38.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.830+0000 7f925d39b640 1 --2- 172.21.15.19:0/2775280969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9258103b70 0x7f925810a400 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.830+0000 7f925d39b640 1 --2- 172.21.15.19:0/2775280969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92581031b0 0x7f9258103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.830+0000 7f925d39b640 1 --2- 172.21.15.19:0/2775280969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9258101fc0 0x7f92581023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.830+0000 7f925d39b640 1 -- 172.21.15.19:0/2775280969 >> 172.21.15.19:0/2775280969 conn(0x7f92580fd550 msgr2=0x7f92580ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.830+0000 7f925d39b640 1 -- 172.21.15.19:0/2775280969 shutdown_connections 2024-04-25T12:04:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.830+0000 7f925d39b640 1 -- 172.21.15.19:0/2775280969 wait complete. 2024-04-25T12:04:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.830+0000 7f925d39b640 1 Processor -- start 2024-04-25T12:04:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.830+0000 7f925d39b640 1 -- start start 2024-04-25T12:04:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f925d39b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9258101fc0 0x7f9258198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f925d39b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92581031b0 0x7f92581987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f925d39b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9258103b70 0x7f925819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:38.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f925d39b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f925810dc20 con 0x7f9258101fc0 2024-04-25T12:04:38.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f925d39b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f925810daa0 con 0x7f9258103b70 2024-04-25T12:04:38.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f925d39b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f925810dda0 con 0x7f92581031b0 2024-04-25T12:04:38.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f9256ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9258101fc0 0x7f9258198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:38.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f9256ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9258101fc0 0x7f9258198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53068/0 (socket says 172.21.15.19:53068) 2024-04-25T12:04:38.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f9256ffd640 1 -- 172.21.15.19:0/921075471 learned_addr learned my addr 172.21.15.19:0/921075471 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:38.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f92567fc640 1 --2- 172.21.15.19:0/921075471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92581031b0 0x7f92581987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:38.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f92577fe640 1 --2- 172.21.15.19:0/921075471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9258103b70 0x7f925819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:38.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f9256ffd640 1 -- 172.21.15.19:0/921075471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92581031b0 msgr2=0x7f92581987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:38.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f9256ffd640 1 --2- 172.21.15.19:0/921075471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92581031b0 0x7f92581987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:38.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f9256ffd640 1 -- 172.21.15.19:0/921075471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9258103b70 msgr2=0x7f925819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f9256ffd640 1 --2- 172.21.15.19:0/921075471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9258103b70 0x7f925819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f9256ffd640 1 -- 172.21.15.19:0/921075471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925819fe80 con 0x7f9258101fc0 2024-04-25T12:04:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f9256ffd640 1 --2- 172.21.15.19:0/921075471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9258101fc0 0x7f9258198260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f924800d950 tx=0x7f924800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f923ffff640 1 -- 172.21.15.19:0/921075471 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248017070 con 0x7f9258101fc0 2024-04-25T12:04:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f923ffff640 1 -- 172.21.15.19:0/921075471 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9248009c30 con 0x7f9258101fc0 2024-04-25T12:04:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f925d39b640 1 -- 172.21.15.19:0/921075471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92581a0170 con 0x7f9258101fc0 2024-04-25T12:04:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f923ffff640 1 -- 172.21.15.19:0/921075471 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924800f620 con 0x7f9258101fc0 2024-04-25T12:04:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.831+0000 7f925d39b640 1 -- 172.21.15.19:0/921075471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92581a0670 con 0x7f9258101fc0 2024-04-25T12:04:38.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.833+0000 7f925d39b640 1 -- 172.21.15.19:0/921075471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9258109280 con 0x7f9258101fc0 2024-04-25T12:04:38.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.834+0000 7f923ffff640 1 -- 172.21.15.19:0/921075471 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f924801a070 con 0x7f9258101fc0 2024-04-25T12:04:38.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.835+0000 7f923ffff640 1 --2- 172.21.15.19:0/921075471 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f92400a3bb0 0x7f92400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:38.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.835+0000 7f92567fc640 1 --2- 172.21.15.19:0/921075471 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f92400a3bb0 0x7f92400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:38.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.835+0000 7f923ffff640 1 -- 172.21.15.19:0/921075471 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f92480cf980 con 0x7f9258101fc0 2024-04-25T12:04:38.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.835+0000 7f92567fc640 1 --2- 172.21.15.19:0/921075471 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f92400a3bb0 0x7f92400a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9258199770 tx=0x7f924c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:38.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:38.839+0000 7f923ffff640 1 -- 172.21.15.19:0/921075471 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9248009de0 con 0x7f9258101fc0 2024-04-25T12:04:39.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:38 smithi019 ceph-mon[29213]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:39.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:38 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.150:6834/1405540140,v1:172.21.15.150:6835/1405540140] up:boot 2024-04-25T12:04:39.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:38 smithi019 ceph-mon[29213]: fsmap 5 up:standby 2024-04-25T12:04:39.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T12:04:39.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:39.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:39.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:39.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:39.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:39.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:38 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:39.066+0000 7f925d39b640 1 -- 172.21.15.19:0/921075471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f925802d5c0 con 0x7f9258101fc0 2024-04-25T12:04:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:39.067+0000 7f923ffff640 1 -- 172.21.15.19:0/921075471 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7f924809c800 con 0x7f9258101fc0 2024-04-25T12:04:39.065 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:39.065 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T12:04:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 6 2024-04-25T12:04:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:39.069+0000 7f925d39b640 1 -- 172.21.15.19:0/921075471 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f92400a3bb0 msgr2=0x7f92400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:39.069+0000 7f925d39b640 1 --2- 172.21.15.19:0/921075471 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f92400a3bb0 0x7f92400a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9258199770 tx=0x7f924c00b410 comp rx=0 tx=0).stop 2024-04-25T12:04:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:39.069+0000 7f925d39b640 1 -- 172.21.15.19:0/921075471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9258101fc0 msgr2=0x7f9258198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:39.069+0000 7f925d39b640 1 --2- 172.21.15.19:0/921075471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9258101fc0 0x7f9258198260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f924800d950 tx=0x7f924800de10 comp rx=0 tx=0).stop 2024-04-25T12:04:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:39.069+0000 7f925d39b640 1 -- 172.21.15.19:0/921075471 shutdown_connections 2024-04-25T12:04:39.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:39.069+0000 7f925d39b640 1 --2- 172.21.15.19:0/921075471 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f92400a3bb0 0x7f92400a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:39.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:39.069+0000 7f925d39b640 1 --2- 172.21.15.19:0/921075471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9258103b70 0x7f925819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:39.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:39.069+0000 7f925d39b640 1 --2- 172.21.15.19:0/921075471 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92581031b0 0x7f92581987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:39.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:39.069+0000 7f925d39b640 1 --2- 172.21.15.19:0/921075471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9258101fc0 0x7f9258198260 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:39.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:39.069+0000 7f925d39b640 1 -- 172.21.15.19:0/921075471 >> 172.21.15.19:0/921075471 conn(0x7f92580fd550 msgr2=0x7f9258106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:39.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:39.069+0000 7f925d39b640 1 -- 172.21.15.19:0/921075471 shutdown_connections 2024-04-25T12:04:39.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:39.069+0000 7f925d39b640 1 -- 172.21.15.19:0/921075471 wait complete. 2024-04-25T12:04:39.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:38 smithi169 ceph-mon[32710]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:39.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:38 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.150:6834/1405540140,v1:172.21.15.150:6835/1405540140] up:boot 2024-04-25T12:04:39.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:38 smithi169 ceph-mon[32710]: fsmap 5 up:standby 2024-04-25T12:04:39.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T12:04:39.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:39.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:39.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:39.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:39.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:39.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:38 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:39.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:38 smithi150 ceph-mon[32250]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:39.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:38 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.150:6834/1405540140,v1:172.21.15.150:6835/1405540140] up:boot 2024-04-25T12:04:39.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:38 smithi150 ceph-mon[32250]: fsmap 5 up:standby 2024-04-25T12:04:39.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T12:04:39.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:39.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:39.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:39.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:39.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:39.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:38 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:39.790 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:39 smithi169 ceph-mon[32710]: Deploying daemon mds.f on smithi169 2024-04-25T12:04:39.791 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:39 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/921075471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:40.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:39 smithi019 ceph-mon[29213]: Deploying daemon mds.f on smithi169 2024-04-25T12:04:40.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:39 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/921075471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:40.155 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:04:39 smithi169 systemd[1]: Starting Ceph mds.f for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:04:40.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:39 smithi150 ceph-mon[32250]: Deploying daemon mds.f on smithi169 2024-04-25T12:04:40.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:39 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/921075471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:40.521 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:04:40 smithi169 podman[63843]: 2024-04-25 12:04:40.101440628 +0000 UTC m=+0.018041584 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:04:40.521 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:04:40 smithi169 podman[63843]: 2024-04-25 12:04:40.221867207 +0000 UTC m=+0.138468148 container create 3ac46dda412ee732e92b9022e67fa2ed858d76754923e13d40ad1e4d109795f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T12:04:40.521 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:04:40 smithi169 podman[63843]: 2024-04-25 12:04:40.524251138 +0000 UTC m=+0.440852081 container init 3ac46dda412ee732e92b9022e67fa2ed858d76754923e13d40ad1e4d109795f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-25T12:04:40.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:40 smithi169 ceph-mon[32710]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:40.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:40.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:40.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:40.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:40.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:40.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:40 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:40.906 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:04:40 smithi169 podman[63843]: 2024-04-25 12:04:40.527000487 +0000 UTC m=+0.443601437 container start 3ac46dda412ee732e92b9022e67fa2ed858d76754923e13d40ad1e4d109795f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T12:04:40.906 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:04:40 smithi169 ceph-mds[63863]: -- 172.21.15.169:0/3678914493 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dfe9d7fa40 con 0x55dfe9d53000 2024-04-25T12:04:40.906 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:04:40 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-f[63859]: starting mds.f at 2024-04-25T12:04:40.906 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:04:40 smithi169 bash[63843]: 3ac46dda412ee732e92b9022e67fa2ed858d76754923e13d40ad1e4d109795f9 2024-04-25T12:04:40.906 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:04:40 smithi169 systemd[1]: Started Ceph mds.f for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T12:04:41.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:40 smithi019 ceph-mon[29213]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:41.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:40 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:41.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:40 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:41.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:40 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:41.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:40 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:41.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:40 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:41.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:40 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:41.079 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:41.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:40 smithi150 ceph-mon[32250]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:41.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:40 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:41.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:40 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:41.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:40 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:41.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:40 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:41.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:40 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:41.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:40 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:41.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.197+0000 7f8e61968640 1 -- 172.21.15.19:0/387502290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c101fc0 msgr2=0x7f8e5c1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:41.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.197+0000 7f8e61968640 1 --2- 172.21.15.19:0/387502290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c101fc0 0x7f8e5c1023e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009900 tx=0x7f8e4c02a410 comp rx=0 tx=0).stop 2024-04-25T12:04:41.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.197+0000 7f8e61968640 1 -- 172.21.15.19:0/387502290 shutdown_connections 2024-04-25T12:04:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.197+0000 7f8e61968640 1 --2- 172.21.15.19:0/387502290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e5c103b70 0x7f8e5c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.197+0000 7f8e61968640 1 --2- 172.21.15.19:0/387502290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e5c1031b0 0x7f8e5c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.197+0000 7f8e61968640 1 --2- 172.21.15.19:0/387502290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c101fc0 0x7f8e5c1023e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.197+0000 7f8e61968640 1 -- 172.21.15.19:0/387502290 >> 172.21.15.19:0/387502290 conn(0x7f8e5c0fd550 msgr2=0x7f8e5c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.197+0000 7f8e61968640 1 -- 172.21.15.19:0/387502290 shutdown_connections 2024-04-25T12:04:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.197+0000 7f8e61968640 1 -- 172.21.15.19:0/387502290 wait complete. 2024-04-25T12:04:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.197+0000 7f8e61968640 1 Processor -- start 2024-04-25T12:04:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.197+0000 7f8e61968640 1 -- start start 2024-04-25T12:04:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.197+0000 7f8e61968640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e5c101fc0 0x7f8e5c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e61968640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e5c1031b0 0x7f8e5c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e61968640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c103b70 0x7f8e5c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:41.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e61968640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c10dc00 con 0x7f8e5c103b70 2024-04-25T12:04:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e61968640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c10da80 con 0x7f8e5c101fc0 2024-04-25T12:04:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e61968640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c10dd80 con 0x7f8e5c1031b0 2024-04-25T12:04:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c103b70 0x7f8e5c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c103b70 0x7f8e5c19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53084/0 (socket says 172.21.15.19:53084) 2024-04-25T12:04:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e53fff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e5c1031b0 0x7f8e5c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e5b7fe640 1 -- 172.21.15.19:0/3594349987 learned_addr learned my addr 172.21.15.19:0/3594349987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e5affd640 1 --2- 172.21.15.19:0/3594349987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e5c101fc0 0x7f8e5c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e5b7fe640 1 -- 172.21.15.19:0/3594349987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e5c1031b0 msgr2=0x7f8e5c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e5b7fe640 1 --2- 172.21.15.19:0/3594349987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e5c1031b0 0x7f8e5c1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e5b7fe640 1 -- 172.21.15.19:0/3594349987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e5c101fc0 msgr2=0x7f8e5c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:41.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e5b7fe640 1 --2- 172.21.15.19:0/3594349987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e5c101fc0 0x7f8e5c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:41.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e5b7fe640 1 -- 172.21.15.19:0/3594349987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5c19fec0 con 0x7f8e5c103b70 2024-04-25T12:04:41.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e5affd640 1 --2- 172.21.15.19:0/3594349987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e5c101fc0 0x7f8e5c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:04:41.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e5b7fe640 1 --2- 172.21.15.19:0/3594349987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c103b70 0x7f8e5c19f850 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400eb10 tx=0x7f8e5400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:41.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e60966640 1 -- 172.21.15.19:0/3594349987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54004b20 con 0x7f8e5c103b70 2024-04-25T12:04:41.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e61968640 1 -- 172.21.15.19:0/3594349987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5c1a01b0 con 0x7f8e5c103b70 2024-04-25T12:04:41.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e60966640 1 -- 172.21.15.19:0/3594349987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8e54004cc0 con 0x7f8e5c103b70 2024-04-25T12:04:41.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e60966640 1 -- 172.21.15.19:0/3594349987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54010720 con 0x7f8e5c103b70 2024-04-25T12:04:41.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.198+0000 7f8e61968640 1 -- 172.21.15.19:0/3594349987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e5c1a0700 con 0x7f8e5c103b70 2024-04-25T12:04:41.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.200+0000 7f8e61968640 1 -- 172.21.15.19:0/3594349987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e5c109360 con 0x7f8e5c103b70 2024-04-25T12:04:41.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.201+0000 7f8e60966640 1 -- 172.21.15.19:0/3594349987 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8e54005020 con 0x7f8e5c103b70 2024-04-25T12:04:41.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.201+0000 7f8e60966640 1 --2- 172.21.15.19:0/3594349987 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8e480a3bb0 0x7f8e480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:41.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.201+0000 7f8e5affd640 1 --2- 172.21.15.19:0/3594349987 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8e480a3bb0 0x7f8e480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:41.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.201+0000 7f8e60966640 1 -- 172.21.15.19:0/3594349987 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8e540d05a0 con 0x7f8e5c103b70 2024-04-25T12:04:41.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.202+0000 7f8e5affd640 1 --2- 172.21.15.19:0/3594349987 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8e480a3bb0 0x7f8e480a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c000c00 tx=0x7f8e4c031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:41.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.206+0000 7f8e60966640 1 -- 172.21.15.19:0/3594349987 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e5409d420 con 0x7f8e5c103b70 2024-04-25T12:04:41.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.432+0000 7f8e61968640 1 -- 172.21.15.19:0/3594349987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e5c062b50 con 0x7f8e5c103b70 2024-04-25T12:04:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.432+0000 7f8e60966640 1 -- 172.21.15.19:0/3594349987 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f8e5409cd20 con 0x7f8e5c103b70 2024-04-25T12:04:41.446 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:41.446 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T12:04:41.447 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 7 2024-04-25T12:04:41.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.434+0000 7f8e61968640 1 -- 172.21.15.19:0/3594349987 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8e480a3bb0 msgr2=0x7f8e480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:41.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.434+0000 7f8e61968640 1 --2- 172.21.15.19:0/3594349987 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8e480a3bb0 0x7f8e480a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c000c00 tx=0x7f8e4c031510 comp rx=0 tx=0).stop 2024-04-25T12:04:41.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.434+0000 7f8e61968640 1 -- 172.21.15.19:0/3594349987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c103b70 msgr2=0x7f8e5c19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:41.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.434+0000 7f8e61968640 1 --2- 172.21.15.19:0/3594349987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c103b70 0x7f8e5c19f850 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400eb10 tx=0x7f8e5400c2d0 comp rx=0 tx=0).stop 2024-04-25T12:04:41.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.434+0000 7f8e61968640 1 -- 172.21.15.19:0/3594349987 shutdown_connections 2024-04-25T12:04:41.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.434+0000 7f8e61968640 1 --2- 172.21.15.19:0/3594349987 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8e480a3bb0 0x7f8e480a6070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:41.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.434+0000 7f8e61968640 1 --2- 172.21.15.19:0/3594349987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e5c103b70 0x7f8e5c19f850 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:41.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.434+0000 7f8e61968640 1 --2- 172.21.15.19:0/3594349987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e5c1031b0 0x7f8e5c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:41.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.434+0000 7f8e61968640 1 --2- 172.21.15.19:0/3594349987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e5c101fc0 0x7f8e5c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:41.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.434+0000 7f8e61968640 1 -- 172.21.15.19:0/3594349987 >> 172.21.15.19:0/3594349987 conn(0x7f8e5c0fd550 msgr2=0x7f8e5c106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:41.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.434+0000 7f8e61968640 1 -- 172.21.15.19:0/3594349987 shutdown_connections 2024-04-25T12:04:41.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:41.434+0000 7f8e61968640 1 -- 172.21.15.19:0/3594349987 wait complete. 2024-04-25T12:04:41.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:41 smithi169 ceph-mon[32710]: Deploying daemon mds.c on smithi169 2024-04-25T12:04:41.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:41 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] up:boot 2024-04-25T12:04:41.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:41 smithi169 ceph-mon[32710]: fsmap 6 up:standby 2024-04-25T12:04:41.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:41 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T12:04:41.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:41 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3594349987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:42.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:41 smithi019 ceph-mon[29213]: Deploying daemon mds.c on smithi169 2024-04-25T12:04:42.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:41 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] up:boot 2024-04-25T12:04:42.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:41 smithi019 ceph-mon[29213]: fsmap 6 up:standby 2024-04-25T12:04:42.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:41 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T12:04:42.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:41 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3594349987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:42.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:41 smithi150 ceph-mon[32250]: Deploying daemon mds.c on smithi169 2024-04-25T12:04:42.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:41 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] up:boot 2024-04-25T12:04:42.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:41 smithi150 ceph-mon[32250]: fsmap 6 up:standby 2024-04-25T12:04:42.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:41 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T12:04:42.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:41 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3594349987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:43.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:42 smithi019 ceph-mon[29213]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:43.069 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:42 smithi169 ceph-mon[32710]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:43.069 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:04:42 smithi169 systemd[1]: Starting Ceph mds.c for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:04:43.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:42 smithi150 ceph-mon[32250]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:43.344 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:04:43 smithi169 podman[64149]: 2024-04-25 12:04:42.974603815 +0000 UTC m=+0.018010925 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:04:43.344 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:04:43 smithi169 podman[64149]: 2024-04-25 12:04:43.102243217 +0000 UTC m=+0.145650317 container create c14b15ecb108210460c1f947f4647727a0c28787ca47bb4fdf5f3f3e925ceea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T12:04:43.344 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:04:43 smithi169 podman[64149]: 2024-04-25 12:04:43.346874246 +0000 UTC m=+0.390281346 container init c14b15ecb108210460c1f947f4647727a0c28787ca47bb4fdf5f3f3e925ceea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:04:43.450 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:43.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.570+0000 7fd4a597d640 1 -- 172.21.15.19:0/2037772970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a0100e90 msgr2=0x7fd4a01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:43.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.570+0000 7fd4a597d640 1 --2- 172.21.15.19:0/2037772970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a0100e90 0x7fd4a01012b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd494009900 tx=0x7fd49402a410 comp rx=0 tx=0).stop 2024-04-25T12:04:43.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.570+0000 7fd4a597d640 1 -- 172.21.15.19:0/2037772970 shutdown_connections 2024-04-25T12:04:43.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.570+0000 7fd4a597d640 1 --2- 172.21.15.19:0/2037772970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a01029a0 0x7fd4a01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:43.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.570+0000 7fd4a597d640 1 --2- 172.21.15.19:0/2037772970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a0101fe0 0x7fd4a0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:43.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.570+0000 7fd4a597d640 1 --2- 172.21.15.19:0/2037772970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a0100e90 0x7fd4a01012b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:43.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.570+0000 7fd4a597d640 1 -- 172.21.15.19:0/2037772970 >> 172.21.15.19:0/2037772970 conn(0x7fd4a00fc400 msgr2=0x7fd4a00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:43.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.570+0000 7fd4a597d640 1 -- 172.21.15.19:0/2037772970 shutdown_connections 2024-04-25T12:04:43.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.570+0000 7fd4a597d640 1 -- 172.21.15.19:0/2037772970 wait complete. 2024-04-25T12:04:43.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.570+0000 7fd4a597d640 1 Processor -- start 2024-04-25T12:04:43.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd4a597d640 1 -- start start 2024-04-25T12:04:43.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd4a597d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a0100e90 0x7fd4a01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:43.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd4a597d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a0101fe0 0x7fd4a0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:43.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd4a597d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a01029a0 0x7fd4a019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd4a597d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a010cb20 con 0x7fd4a0100e90 2024-04-25T12:04:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd4a597d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a010c9a0 con 0x7fd4a01029a0 2024-04-25T12:04:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd4a597d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a010cca0 con 0x7fd4a0101fe0 2024-04-25T12:04:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd49effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a0100e90 0x7fd4a01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd49effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a0100e90 0x7fd4a01982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56780/0 (socket says 172.21.15.19:56780) 2024-04-25T12:04:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd49effd640 1 -- 172.21.15.19:0/2722885841 learned_addr learned my addr 172.21.15.19:0/2722885841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd49f7fe640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a01029a0 0x7fd4a019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd49e7fc640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a0101fe0 0x7fd4a0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd49effd640 1 -- 172.21.15.19:0/2722885841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a0101fe0 msgr2=0x7fd4a0198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd49effd640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a0101fe0 0x7fd4a0198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd49effd640 1 -- 172.21.15.19:0/2722885841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a01029a0 msgr2=0x7fd4a019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd49effd640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a01029a0 0x7fd4a019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:43.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd49effd640 1 -- 172.21.15.19:0/2722885841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a019ff10 con 0x7fd4a0100e90 2024-04-25T12:04:43.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd49e7fc640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a0101fe0 0x7fd4a0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:04:43.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.571+0000 7fd49effd640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a0100e90 0x7fd4a01982f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd494005940 tx=0x7fd4940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:43.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.572+0000 7fd4a497b640 1 -- 172.21.15.19:0/2722885841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49403f030 con 0x7fd4a0100e90 2024-04-25T12:04:43.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.572+0000 7fd4a497b640 1 -- 172.21.15.19:0/2722885841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd494003cd0 con 0x7fd4a0100e90 2024-04-25T12:04:43.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.572+0000 7fd4a497b640 1 -- 172.21.15.19:0/2722885841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494035070 con 0x7fd4a0100e90 2024-04-25T12:04:43.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.572+0000 7fd4a597d640 1 -- 172.21.15.19:0/2722885841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a01a01a0 con 0x7fd4a0100e90 2024-04-25T12:04:43.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.572+0000 7fd4a597d640 1 -- 172.21.15.19:0/2722885841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a01a0690 con 0x7fd4a0100e90 2024-04-25T12:04:43.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.573+0000 7fd4a597d640 1 -- 172.21.15.19:0/2722885841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a01a08f0 con 0x7fd4a0100e90 2024-04-25T12:04:43.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.574+0000 7fd4a497b640 1 -- 172.21.15.19:0/2722885841 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd494042030 con 0x7fd4a0100e90 2024-04-25T12:04:43.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.575+0000 7fd4a497b640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd4880a3bb0 0x7fd4880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:43.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.575+0000 7fd49e7fc640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd4880a3bb0 0x7fd4880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:43.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.575+0000 7fd4a497b640 1 -- 172.21.15.19:0/2722885841 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd4940f04f0 con 0x7fd4a0100e90 2024-04-25T12:04:43.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.575+0000 7fd49e7fc640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd4880a3bb0 0x7fd4880a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0199800 tx=0x7fd490005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:43.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.580+0000 7fd4a497b640 1 -- 172.21.15.19:0/2722885841 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4940bd2d0 con 0x7fd4a0100e90 2024-04-25T12:04:43.655 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:04:43 smithi169 podman[64149]: 2024-04-25 12:04:43.349419236 +0000 UTC m=+0.392826344 container start c14b15ecb108210460c1f947f4647727a0c28787ca47bb4fdf5f3f3e925ceea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T12:04:43.655 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:04:43 smithi169 ceph-mds[64168]: -- 172.21.15.169:0/4211723801 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9f79bda40 con 0x55e9f7991000 2024-04-25T12:04:43.655 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:04:43 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-c[64164]: starting mds.c at 2024-04-25T12:04:43.655 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:04:43 smithi169 bash[64149]: c14b15ecb108210460c1f947f4647727a0c28787ca47bb4fdf5f3f3e925ceea6 2024-04-25T12:04:43.655 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:04:43 smithi169 systemd[1]: Started Ceph mds.c for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T12:04:43.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.806+0000 7fd4a597d640 1 -- 172.21.15.19:0/2722885841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4a01a0ac0 con 0x7fd4a0100e90 2024-04-25T12:04:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.807+0000 7fd4a497b640 1 -- 172.21.15.19:0/2722885841 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7fd4940bcbd0 con 0x7fd4a0100e90 2024-04-25T12:04:43.805 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:43.805 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T12:04:43.805 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-04-25T12:04:43.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.809+0000 7fd4a597d640 1 -- 172.21.15.19:0/2722885841 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd4880a3bb0 msgr2=0x7fd4880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:43.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.809+0000 7fd4a597d640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd4880a3bb0 0x7fd4880a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0199800 tx=0x7fd490005f90 comp rx=0 tx=0).stop 2024-04-25T12:04:43.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.809+0000 7fd4a597d640 1 -- 172.21.15.19:0/2722885841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a0100e90 msgr2=0x7fd4a01982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:43.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.809+0000 7fd4a597d640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a0100e90 0x7fd4a01982f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd494005940 tx=0x7fd4940042f0 comp rx=0 tx=0).stop 2024-04-25T12:04:43.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.809+0000 7fd4a597d640 1 -- 172.21.15.19:0/2722885841 shutdown_connections 2024-04-25T12:04:43.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.809+0000 7fd4a597d640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd4880a3bb0 0x7fd4880a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:43.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.809+0000 7fd4a597d640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a01029a0 0x7fd4a019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.809+0000 7fd4a597d640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd4a0101fe0 0x7fd4a0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.809+0000 7fd4a597d640 1 --2- 172.21.15.19:0/2722885841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4a0100e90 0x7fd4a01982f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.809+0000 7fd4a597d640 1 -- 172.21.15.19:0/2722885841 >> 172.21.15.19:0/2722885841 conn(0x7fd4a00fc400 msgr2=0x7fd4a00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.809+0000 7fd4a597d640 1 -- 172.21.15.19:0/2722885841 shutdown_connections 2024-04-25T12:04:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:43.809+0000 7fd4a597d640 1 -- 172.21.15.19:0/2722885841 wait complete. 2024-04-25T12:04:44.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:44 smithi019 ceph-mon[29213]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:44.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:44 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:44.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:44 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:44.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:44 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:44.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:44 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:44.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:44 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:44.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:44 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:44.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:44 smithi019 ceph-mon[29213]: Deploying daemon mds.h on smithi150 2024-04-25T12:04:44.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:44 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] up:boot 2024-04-25T12:04:44.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:44 smithi019 ceph-mon[29213]: fsmap 7 up:standby 2024-04-25T12:04:44.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:44 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T12:04:44.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:44 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2722885841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:44.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:44 smithi169 ceph-mon[32710]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:44.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:44 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:44.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:44 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:44.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:44 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:44.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:44 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:44.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:44 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:44.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:44 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:44.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:44 smithi169 ceph-mon[32710]: Deploying daemon mds.h on smithi150 2024-04-25T12:04:44.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:44 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] up:boot 2024-04-25T12:04:44.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:44 smithi169 ceph-mon[32710]: fsmap 7 up:standby 2024-04-25T12:04:44.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:44 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T12:04:44.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:44 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2722885841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:44.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:44 smithi150 ceph-mon[32250]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:44.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:44 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:44.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:44 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:44.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:44 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:44.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:44 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:44.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:44 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:44.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:44 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:44.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:44 smithi150 ceph-mon[32250]: Deploying daemon mds.h on smithi150 2024-04-25T12:04:44.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:44 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] up:boot 2024-04-25T12:04:44.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:44 smithi150 ceph-mon[32250]: fsmap 7 up:standby 2024-04-25T12:04:44.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:44 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T12:04:44.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:44 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2722885841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:45.822 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:45.913 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:04:45 smithi150 systemd[1]: Starting Ceph mds.h for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:04:45.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.939+0000 7fda0adad640 1 -- 172.21.15.19:0/1763187459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda04103190 msgr2=0x7fda04103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:45.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.939+0000 7fda0adad640 1 --2- 172.21.15.19:0/1763187459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda04103190 0x7fda04103610 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009900 tx=0x7fd9f002a410 comp rx=0 tx=0).stop 2024-04-25T12:04:45.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.939+0000 7fda0adad640 1 -- 172.21.15.19:0/1763187459 shutdown_connections 2024-04-25T12:04:45.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.939+0000 7fda0adad640 1 --2- 172.21.15.19:0/1763187459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda04103b50 0x7fda0410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:45.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.939+0000 7fda0adad640 1 --2- 172.21.15.19:0/1763187459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda04103190 0x7fda04103610 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:45.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.939+0000 7fda0adad640 1 --2- 172.21.15.19:0/1763187459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda04101fa0 0x7fda041023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:45.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.939+0000 7fda0adad640 1 -- 172.21.15.19:0/1763187459 >> 172.21.15.19:0/1763187459 conn(0x7fda040fd550 msgr2=0x7fda040ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:45.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.939+0000 7fda0adad640 1 -- 172.21.15.19:0/1763187459 shutdown_connections 2024-04-25T12:04:45.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.939+0000 7fda0adad640 1 -- 172.21.15.19:0/1763187459 wait complete. 2024-04-25T12:04:45.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda0adad640 1 Processor -- start 2024-04-25T12:04:45.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda0adad640 1 -- start start 2024-04-25T12:04:45.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda0adad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda04101fa0 0x7fda04075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:45.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda0adad640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda04103190 0x7fda04074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:45.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda0adad640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda04103b50 0x7fda04074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda0adad640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda0410dc00 con 0x7fda04101fa0 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda0adad640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fda0410da80 con 0x7fda04103190 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda0adad640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fda0410dd80 con 0x7fda04103b50 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda08b49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda04101fa0 0x7fda04075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda08b49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda04101fa0 0x7fda04075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56800/0 (socket says 172.21.15.19:56800) 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda08b49640 1 -- 172.21.15.19:0/3300094589 learned_addr learned my addr 172.21.15.19:0/3300094589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda03fff640 1 --2- 172.21.15.19:0/3300094589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda04103190 0x7fda04074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda0934a640 1 --2- 172.21.15.19:0/3300094589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda04103b50 0x7fda04074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda08b49640 1 -- 172.21.15.19:0/3300094589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda04103b50 msgr2=0x7fda04074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.940+0000 7fda08b49640 1 --2- 172.21.15.19:0/3300094589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda04103b50 0x7fda04074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.941+0000 7fda08b49640 1 -- 172.21.15.19:0/3300094589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda04103190 msgr2=0x7fda04074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.941+0000 7fda08b49640 1 --2- 172.21.15.19:0/3300094589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda04103190 0x7fda04074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.941+0000 7fda08b49640 1 -- 172.21.15.19:0/3300094589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda04075110 con 0x7fda04101fa0 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.941+0000 7fda08b49640 1 --2- 172.21.15.19:0/3300094589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda04101fa0 0x7fda04075d00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800e9c0 tx=0x7fd9f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:45.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.941+0000 7fda01ffb640 1 -- 172.21.15.19:0/3300094589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f800cd80 con 0x7fda04101fa0 2024-04-25T12:04:45.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.941+0000 7fda0adad640 1 -- 172.21.15.19:0/3300094589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda041a23c0 con 0x7fda04101fa0 2024-04-25T12:04:45.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.941+0000 7fda01ffb640 1 -- 172.21.15.19:0/3300094589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd9f80035f0 con 0x7fda04101fa0 2024-04-25T12:04:45.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.941+0000 7fda01ffb640 1 -- 172.21.15.19:0/3300094589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8010690 con 0x7fda04101fa0 2024-04-25T12:04:45.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.941+0000 7fda0adad640 1 -- 172.21.15.19:0/3300094589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda041a28e0 con 0x7fda04101fa0 2024-04-25T12:04:45.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.942+0000 7fda0adad640 1 -- 172.21.15.19:0/3300094589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda040691c0 con 0x7fda04101fa0 2024-04-25T12:04:45.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.943+0000 7fda01ffb640 1 -- 172.21.15.19:0/3300094589 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd9f80037a0 con 0x7fda04101fa0 2024-04-25T12:04:45.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.944+0000 7fda01ffb640 1 --2- 172.21.15.19:0/3300094589 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd9f40a3bb0 0x7fd9f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:45.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.944+0000 7fda03fff640 1 --2- 172.21.15.19:0/3300094589 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd9f40a3bb0 0x7fd9f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:45.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.944+0000 7fda01ffb640 1 -- 172.21.15.19:0/3300094589 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd9f80d0460 con 0x7fda04101fa0 2024-04-25T12:04:45.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.944+0000 7fda03fff640 1 --2- 172.21.15.19:0/3300094589 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd9f40a3bb0 0x7fd9f40a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0000c00 tx=0x7fd9f0031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:45.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:45.948+0000 7fda01ffb640 1 -- 172.21.15.19:0/3300094589 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9f809d240 con 0x7fda04101fa0 2024-04-25T12:04:46.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:46.176+0000 7fda0adad640 1 -- 172.21.15.19:0/3300094589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda04109390 con 0x7fda04101fa0 2024-04-25T12:04:46.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:46.176+0000 7fda01ffb640 1 -- 172.21.15.19:0/3300094589 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7fda04109390 con 0x7fda04101fa0 2024-04-25T12:04:46.175 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:46.175 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T12:04:46.175 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-04-25T12:04:46.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:46.178+0000 7fda0adad640 1 -- 172.21.15.19:0/3300094589 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd9f40a3bb0 msgr2=0x7fd9f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:46.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:46.178+0000 7fda0adad640 1 --2- 172.21.15.19:0/3300094589 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd9f40a3bb0 0x7fd9f40a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0000c00 tx=0x7fd9f0031510 comp rx=0 tx=0).stop 2024-04-25T12:04:46.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:46.178+0000 7fda0adad640 1 -- 172.21.15.19:0/3300094589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda04101fa0 msgr2=0x7fda04075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:46.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:46.178+0000 7fda0adad640 1 --2- 172.21.15.19:0/3300094589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda04101fa0 0x7fda04075d00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800e9c0 tx=0x7fd9f800ee80 comp rx=0 tx=0).stop 2024-04-25T12:04:46.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:46.179+0000 7fda0adad640 1 -- 172.21.15.19:0/3300094589 shutdown_connections 2024-04-25T12:04:46.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:46.179+0000 7fda0adad640 1 --2- 172.21.15.19:0/3300094589 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd9f40a3bb0 0x7fd9f40a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:46.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:46.179+0000 7fda0adad640 1 --2- 172.21.15.19:0/3300094589 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda04103b50 0x7fda04074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:46.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:46.179+0000 7fda0adad640 1 --2- 172.21.15.19:0/3300094589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda04103190 0x7fda04074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:46.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:46.179+0000 7fda0adad640 1 --2- 172.21.15.19:0/3300094589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda04101fa0 0x7fda04075d00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:46.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:46.179+0000 7fda0adad640 1 -- 172.21.15.19:0/3300094589 >> 172.21.15.19:0/3300094589 conn(0x7fda040fd550 msgr2=0x7fda04106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:46.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:46.179+0000 7fda0adad640 1 -- 172.21.15.19:0/3300094589 shutdown_connections 2024-04-25T12:04:46.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:46.179+0000 7fda0adad640 1 -- 172.21.15.19:0/3300094589 wait complete. 2024-04-25T12:04:46.341 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:04:46 smithi150 podman[62506]: 2024-04-25 12:04:45.960966044 +0000 UTC m=+0.018339844 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:04:46.341 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:04:46 smithi150 podman[62506]: 2024-04-25 12:04:46.099990499 +0000 UTC m=+0.157364288 container create 3c0afcfe116f5b36d7d7217a6d4d84d93792c1ec853c102d0fd589b03cb39403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T12:04:46.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:46 smithi150 ceph-mon[32250]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:46.663 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:46 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3300094589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:46.663 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:04:46 smithi150 podman[62506]: 2024-04-25 12:04:46.344621084 +0000 UTC m=+0.401994872 container init 3c0afcfe116f5b36d7d7217a6d4d84d93792c1ec853c102d0fd589b03cb39403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:04:46.663 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:04:46 smithi150 podman[62506]: 2024-04-25 12:04:46.347326155 +0000 UTC m=+0.404699944 container start 3c0afcfe116f5b36d7d7217a6d4d84d93792c1ec853c102d0fd589b03cb39403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:04:46.663 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:04:46 smithi150 ceph-mds[62525]: -- 172.21.15.150:0/3108401850 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a338e7a40 con 0x561a338ba800 2024-04-25T12:04:46.663 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:04:46 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-h[62521]: starting mds.h at 2024-04-25T12:04:46.663 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:04:46 smithi150 bash[62506]: 3c0afcfe116f5b36d7d7217a6d4d84d93792c1ec853c102d0fd589b03cb39403 2024-04-25T12:04:46.663 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:04:46 smithi150 systemd[1]: Started Ceph mds.h for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T12:04:46.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:46 smithi019 ceph-mon[29213]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:46.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:46 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3300094589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:46.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:46 smithi169 ceph-mon[32710]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:46.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:46 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3300094589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:47.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:47.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:47 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.150:6836/3918801450,v1:172.21.15.150:6837/3918801450] up:boot 2024-04-25T12:04:47.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:47 smithi019 ceph-mon[29213]: fsmap 8 up:standby 2024-04-25T12:04:47.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T12:04:47.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:47.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:47.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:47.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:47.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:47 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:47.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:47 smithi019 ceph-mon[29213]: Deploying daemon mds.i on smithi169 2024-04-25T12:04:47.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:47.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:47 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.150:6836/3918801450,v1:172.21.15.150:6837/3918801450] up:boot 2024-04-25T12:04:47.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:47 smithi169 ceph-mon[32710]: fsmap 8 up:standby 2024-04-25T12:04:47.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T12:04:47.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:47.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:47.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:47.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:47.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:47 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:47.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:47 smithi169 ceph-mon[32710]: Deploying daemon mds.i on smithi169 2024-04-25T12:04:47.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:47.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:47 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.150:6836/3918801450,v1:172.21.15.150:6837/3918801450] up:boot 2024-04-25T12:04:47.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:47 smithi150 ceph-mon[32250]: fsmap 8 up:standby 2024-04-25T12:04:47.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T12:04:47.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:47.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:47.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:47.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:47.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:47 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:47.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:47 smithi150 ceph-mon[32250]: Deploying daemon mds.i on smithi169 2024-04-25T12:04:48.189 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:48.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.307+0000 7f7881ab4640 1 -- 172.21.15.19:0/1609282147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f787c1031b0 msgr2=0x7f787c103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.307+0000 7f7881ab4640 1 --2- 172.21.15.19:0/1609282147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f787c1031b0 0x7f787c103630 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f786c009900 tx=0x7f786c02a410 comp rx=0 tx=0).stop 2024-04-25T12:04:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.307+0000 7f7881ab4640 1 -- 172.21.15.19:0/1609282147 shutdown_connections 2024-04-25T12:04:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.307+0000 7f7881ab4640 1 --2- 172.21.15.19:0/1609282147 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c103b70 0x7f787c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.307+0000 7f7881ab4640 1 --2- 172.21.15.19:0/1609282147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f787c1031b0 0x7f787c103630 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.307+0000 7f7881ab4640 1 --2- 172.21.15.19:0/1609282147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787c101fc0 0x7f787c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.307+0000 7f7881ab4640 1 -- 172.21.15.19:0/1609282147 >> 172.21.15.19:0/1609282147 conn(0x7f787c0fd550 msgr2=0x7f787c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.307+0000 7f7881ab4640 1 -- 172.21.15.19:0/1609282147 shutdown_connections 2024-04-25T12:04:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.307+0000 7f7881ab4640 1 -- 172.21.15.19:0/1609282147 wait complete. 2024-04-25T12:04:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f7881ab4640 1 Processor -- start 2024-04-25T12:04:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f7881ab4640 1 -- start start 2024-04-25T12:04:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f7881ab4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787c101fc0 0x7f787c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f7881ab4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f787c1031b0 0x7f787c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f7881ab4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c103b70 0x7f787c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f7881ab4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f787c10dc20 con 0x7f787c1031b0 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f7881ab4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f787c10daa0 con 0x7f787c101fc0 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f7881ab4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f787c10dda0 con 0x7f787c103b70 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f787affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f787c1031b0 0x7f787c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f787affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f787c1031b0 0x7f787c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56820/0 (socket says 172.21.15.19:56820) 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f787affd640 1 -- 172.21.15.19:0/3978363770 learned_addr learned my addr 172.21.15.19:0/3978363770 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f787bfff640 1 --2- 172.21.15.19:0/3978363770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c103b70 0x7f787c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f787b7fe640 1 --2- 172.21.15.19:0/3978363770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787c101fc0 0x7f787c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f787affd640 1 -- 172.21.15.19:0/3978363770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c103b70 msgr2=0x7f787c19f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f787affd640 1 --2- 172.21.15.19:0/3978363770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c103b70 0x7f787c19f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f787affd640 1 -- 172.21.15.19:0/3978363770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787c101fc0 msgr2=0x7f787c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f787affd640 1 --2- 172.21.15.19:0/3978363770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787c101fc0 0x7f787c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.308+0000 7f787affd640 1 -- 172.21.15.19:0/3978363770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c19fed0 con 0x7f787c1031b0 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.309+0000 7f787affd640 1 --2- 172.21.15.19:0/3978363770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f787c1031b0 0x7f787c1987f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f786c006010 tx=0x7f786c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.309+0000 7f7878ff9640 1 -- 172.21.15.19:0/3978363770 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c03f030 con 0x7f787c1031b0 2024-04-25T12:04:48.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.309+0000 7f7881ab4640 1 -- 172.21.15.19:0/3978363770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787c1a0160 con 0x7f787c1031b0 2024-04-25T12:04:48.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.309+0000 7f7878ff9640 1 -- 172.21.15.19:0/3978363770 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f786c003cd0 con 0x7f787c1031b0 2024-04-25T12:04:48.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.309+0000 7f7878ff9640 1 -- 172.21.15.19:0/3978363770 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c035070 con 0x7f787c1031b0 2024-04-25T12:04:48.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.309+0000 7f7881ab4640 1 -- 172.21.15.19:0/3978363770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787c1a06a0 con 0x7f787c1031b0 2024-04-25T12:04:48.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.310+0000 7f7881ab4640 1 -- 172.21.15.19:0/3978363770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787c109280 con 0x7f787c1031b0 2024-04-25T12:04:48.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.311+0000 7f7878ff9640 1 -- 172.21.15.19:0/3978363770 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f786c042030 con 0x7f787c1031b0 2024-04-25T12:04:48.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.312+0000 7f7878ff9640 1 --2- 172.21.15.19:0/3978363770 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f78680a3bb0 0x7f78680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:48.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.312+0000 7f787b7fe640 1 --2- 172.21.15.19:0/3978363770 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f78680a3bb0 0x7f78680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:48.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.312+0000 7f7878ff9640 1 -- 172.21.15.19:0/3978363770 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f786c0f0940 con 0x7f787c1031b0 2024-04-25T12:04:48.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.312+0000 7f787b7fe640 1 --2- 172.21.15.19:0/3978363770 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f78680a3bb0 0x7f78680a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7864005fd0 tx=0x7f7864005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:48.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.315+0000 7f7878ff9640 1 -- 172.21.15.19:0/3978363770 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f786c0bd720 con 0x7f787c1031b0 2024-04-25T12:04:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.541+0000 7f7881ab4640 1 -- 172.21.15.19:0/3978363770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f787c02d5c0 con 0x7f787c1031b0 2024-04-25T12:04:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.541+0000 7f7878ff9640 1 -- 172.21.15.19:0/3978363770 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7f786c0bd020 con 0x7f787c1031b0 2024-04-25T12:04:48.539 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:48.540 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3918801450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3918801450},{"type":"v1","addr":"172.21.15.150:6837","nonce":3918801450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T12:04:48.540 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-04-25T12:04:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.543+0000 7f7881ab4640 1 -- 172.21.15.19:0/3978363770 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f78680a3bb0 msgr2=0x7f78680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.543+0000 7f7881ab4640 1 --2- 172.21.15.19:0/3978363770 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f78680a3bb0 0x7f78680a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7864005fd0 tx=0x7f7864005850 comp rx=0 tx=0).stop 2024-04-25T12:04:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.543+0000 7f7881ab4640 1 -- 172.21.15.19:0/3978363770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f787c1031b0 msgr2=0x7f787c1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.543+0000 7f7881ab4640 1 --2- 172.21.15.19:0/3978363770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f787c1031b0 0x7f787c1987f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f786c006010 tx=0x7f786c004430 comp rx=0 tx=0).stop 2024-04-25T12:04:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.544+0000 7f7881ab4640 1 -- 172.21.15.19:0/3978363770 shutdown_connections 2024-04-25T12:04:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.544+0000 7f7881ab4640 1 --2- 172.21.15.19:0/3978363770 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f78680a3bb0 0x7f78680a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.544+0000 7f7881ab4640 1 --2- 172.21.15.19:0/3978363770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f787c103b70 0x7f787c19f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.544+0000 7f7881ab4640 1 --2- 172.21.15.19:0/3978363770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f787c1031b0 0x7f787c1987f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:48.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.544+0000 7f7881ab4640 1 --2- 172.21.15.19:0/3978363770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787c101fc0 0x7f787c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:48.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.544+0000 7f7881ab4640 1 -- 172.21.15.19:0/3978363770 >> 172.21.15.19:0/3978363770 conn(0x7f787c0fd550 msgr2=0x7f787c106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:48.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.544+0000 7f7881ab4640 1 -- 172.21.15.19:0/3978363770 shutdown_connections 2024-04-25T12:04:48.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:48.544+0000 7f7881ab4640 1 -- 172.21.15.19:0/3978363770 wait complete. 2024-04-25T12:04:48.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:48 smithi019 ceph-mon[29213]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:48.844 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:48 smithi169 ceph-mon[32710]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:48.844 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:04:48 smithi169 systemd[1]: Starting Ceph mds.i for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:04:48.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:48 smithi150 ceph-mon[32250]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:49.108 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:04:48 smithi169 podman[64455]: 2024-04-25 12:04:48.750474026 +0000 UTC m=+0.018238629 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:04:49.108 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:04:48 smithi169 podman[64455]: 2024-04-25 12:04:48.865211178 +0000 UTC m=+0.132975822 container create bdf8b83936cf5911f193fd9d3232ec80244ab110c365ead3236fa5e318386405 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T12:04:49.108 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:04:49 smithi169 podman[64455]: 2024-04-25 12:04:49.111268123 +0000 UTC m=+0.379032717 container init bdf8b83936cf5911f193fd9d3232ec80244ab110c365ead3236fa5e318386405 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T12:04:49.405 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:04:49 smithi169 podman[64455]: 2024-04-25 12:04:49.114166233 +0000 UTC m=+0.381930824 container start bdf8b83936cf5911f193fd9d3232ec80244ab110c365ead3236fa5e318386405 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T12:04:49.405 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:04:49 smithi169 ceph-mds[64474]: -- 172.21.15.169:0/867573658 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d78467a40 con 0x558d7843b000 2024-04-25T12:04:49.405 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:04:49 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i[64470]: starting mds.i at 2024-04-25T12:04:49.405 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:04:49 smithi169 bash[64455]: bdf8b83936cf5911f193fd9d3232ec80244ab110c365ead3236fa5e318386405 2024-04-25T12:04:49.405 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:04:49 smithi169 systemd[1]: Started Ceph mds.i for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T12:04:49.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:49 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3978363770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:49.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:49.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:49.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:49.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:49.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:49.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:49 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:49.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:49 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3978363770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:49.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:49.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:49.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:49.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:49.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:49.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:49 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:49.918 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:49 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3978363770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:49.918 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:49.919 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:49.919 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:49.919 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:49.919 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:49.919 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:49 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:50.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:50.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.667+0000 7f164c872640 1 -- 172.21.15.19:0/1432267941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648100e90 msgr2=0x7f16481012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:50.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.667+0000 7f164c872640 1 --2- 172.21.15.19:0/1432267941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648100e90 0x7f16481012b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1638009900 tx=0x7f163802a410 comp rx=0 tx=0).stop 2024-04-25T12:04:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.668+0000 7f164c872640 1 -- 172.21.15.19:0/1432267941 shutdown_connections 2024-04-25T12:04:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.668+0000 7f164c872640 1 --2- 172.21.15.19:0/1432267941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16481029a0 0x7f16481092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.668+0000 7f164c872640 1 --2- 172.21.15.19:0/1432267941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1648101fe0 0x7f1648102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.668+0000 7f164c872640 1 --2- 172.21.15.19:0/1432267941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648100e90 0x7f16481012b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.668+0000 7f164c872640 1 -- 172.21.15.19:0/1432267941 >> 172.21.15.19:0/1432267941 conn(0x7f16480fc400 msgr2=0x7f16480fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.668+0000 7f164c872640 1 -- 172.21.15.19:0/1432267941 shutdown_connections 2024-04-25T12:04:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.668+0000 7f164c872640 1 -- 172.21.15.19:0/1432267941 wait complete. 2024-04-25T12:04:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.668+0000 7f164c872640 1 Processor -- start 2024-04-25T12:04:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.668+0000 7f164c872640 1 -- start start 2024-04-25T12:04:50.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f164c872640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1648100e90 0x7f1648198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:50.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f164c872640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648101fe0 0x7f16481987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:50.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f164c872640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16481029a0 0x7f164819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:50.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f164c872640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f164810cb70 con 0x7f1648101fe0 2024-04-25T12:04:50.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f164c872640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f164810c9f0 con 0x7f1648100e90 2024-04-25T12:04:50.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f164c872640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f164810ccf0 con 0x7f16481029a0 2024-04-25T12:04:50.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f1645d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648101fe0 0x7f16481987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:50.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f1645d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648101fe0 0x7f16481987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56848/0 (socket says 172.21.15.19:56848) 2024-04-25T12:04:50.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f1645d9b640 1 -- 172.21.15.19:0/1063626987 learned_addr learned my addr 172.21.15.19:0/1063626987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:50.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f1646d9d640 1 --2- 172.21.15.19:0/1063626987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16481029a0 0x7f164819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:50.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f164659c640 1 --2- 172.21.15.19:0/1063626987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1648100e90 0x7f1648198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:50.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f1645d9b640 1 -- 172.21.15.19:0/1063626987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16481029a0 msgr2=0x7f164819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:50.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f1645d9b640 1 --2- 172.21.15.19:0/1063626987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16481029a0 0x7f164819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:50.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f1645d9b640 1 -- 172.21.15.19:0/1063626987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1648100e90 msgr2=0x7f1648198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:50.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f1645d9b640 1 --2- 172.21.15.19:0/1063626987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1648100e90 0x7f1648198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:50.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f1645d9b640 1 -- 172.21.15.19:0/1063626987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164819fe90 con 0x7f1648101fe0 2024-04-25T12:04:50.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f1645d9b640 1 --2- 172.21.15.19:0/1063626987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648101fe0 0x7f16481987b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f163c00c970 tx=0x7f163c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:50.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f16377fe640 1 -- 172.21.15.19:0/1063626987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c007c10 con 0x7f1648101fe0 2024-04-25T12:04:50.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f16377fe640 1 -- 172.21.15.19:0/1063626987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f163c007db0 con 0x7f1648101fe0 2024-04-25T12:04:50.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f16377fe640 1 -- 172.21.15.19:0/1063626987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c00f910 con 0x7f1648101fe0 2024-04-25T12:04:50.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f164c872640 1 -- 172.21.15.19:0/1063626987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16481a0180 con 0x7f1648101fe0 2024-04-25T12:04:50.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.669+0000 7f164c872640 1 -- 172.21.15.19:0/1063626987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16481a06d0 con 0x7f1648101fe0 2024-04-25T12:04:50.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.670+0000 7f164c872640 1 -- 172.21.15.19:0/1063626987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16481a0990 con 0x7f1648101fe0 2024-04-25T12:04:50.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.672+0000 7f16377fe640 1 -- 172.21.15.19:0/1063626987 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f163c00fab0 con 0x7f1648101fe0 2024-04-25T12:04:50.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.672+0000 7f16377fe640 1 --2- 172.21.15.19:0/1063626987 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f16300a3bb0 0x7f16300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:50.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.672+0000 7f164659c640 1 --2- 172.21.15.19:0/1063626987 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f16300a3bb0 0x7f16300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:50.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.672+0000 7f16377fe640 1 -- 172.21.15.19:0/1063626987 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f163c0d0330 con 0x7f1648101fe0 2024-04-25T12:04:50.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.673+0000 7f164659c640 1 --2- 172.21.15.19:0/1063626987 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f16300a3bb0 0x7f16300a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1638000c00 tx=0x7f1638031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.677+0000 7f16377fe640 1 -- 172.21.15.19:0/1063626987 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f163c09d240 con 0x7f1648101fe0 2024-04-25T12:04:50.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.904+0000 7f164c872640 1 -- 172.21.15.19:0/1063626987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16481a0b60 con 0x7f1648101fe0 2024-04-25T12:04:50.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.904+0000 7f16377fe640 1 -- 172.21.15.19:0/1063626987 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7f163c09cb40 con 0x7f1648101fe0 2024-04-25T12:04:50.903 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:50.903 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3918801450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3918801450},{"type":"v1","addr":"172.21.15.150:6837","nonce":3918801450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/1723439500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1723439500},{"type":"v1","addr":"172.21.15.169:6839","nonce":1723439500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T12:04:50.903 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 10 2024-04-25T12:04:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.906+0000 7f164c872640 1 -- 172.21.15.19:0/1063626987 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f16300a3bb0 msgr2=0x7f16300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.906+0000 7f164c872640 1 --2- 172.21.15.19:0/1063626987 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f16300a3bb0 0x7f16300a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1638000c00 tx=0x7f1638031510 comp rx=0 tx=0).stop 2024-04-25T12:04:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.906+0000 7f164c872640 1 -- 172.21.15.19:0/1063626987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648101fe0 msgr2=0x7f16481987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.906+0000 7f164c872640 1 --2- 172.21.15.19:0/1063626987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648101fe0 0x7f16481987b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f163c00c970 tx=0x7f163c00ce30 comp rx=0 tx=0).stop 2024-04-25T12:04:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.906+0000 7f164c872640 1 -- 172.21.15.19:0/1063626987 shutdown_connections 2024-04-25T12:04:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.906+0000 7f164c872640 1 --2- 172.21.15.19:0/1063626987 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f16300a3bb0 0x7f16300a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.906+0000 7f164c872640 1 --2- 172.21.15.19:0/1063626987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16481029a0 0x7f164819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:50.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.906+0000 7f164c872640 1 --2- 172.21.15.19:0/1063626987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1648101fe0 0x7f16481987b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.906+0000 7f164c872640 1 --2- 172.21.15.19:0/1063626987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1648100e90 0x7f1648198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.906+0000 7f164c872640 1 -- 172.21.15.19:0/1063626987 >> 172.21.15.19:0/1063626987 conn(0x7f16480fc400 msgr2=0x7f16480fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.906+0000 7f164c872640 1 -- 172.21.15.19:0/1063626987 shutdown_connections 2024-04-25T12:04:50.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:50.906+0000 7f164c872640 1 -- 172.21.15.19:0/1063626987 wait complete. 2024-04-25T12:04:50.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:50 smithi169 ceph-mon[32710]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:50.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:50 smithi169 ceph-mon[32710]: Deploying daemon mds.g on smithi019 2024-04-25T12:04:50.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:50 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] up:boot 2024-04-25T12:04:50.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:50 smithi169 ceph-mon[32710]: fsmap 9 up:standby 2024-04-25T12:04:50.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:50 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T12:04:50.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:50 smithi150 ceph-mon[32250]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:50.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:50 smithi150 ceph-mon[32250]: Deploying daemon mds.g on smithi019 2024-04-25T12:04:50.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:50 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] up:boot 2024-04-25T12:04:50.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:50 smithi150 ceph-mon[32250]: fsmap 9 up:standby 2024-04-25T12:04:50.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:50 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T12:04:50.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:50 smithi019 ceph-mon[29213]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:50.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:50 smithi019 ceph-mon[29213]: Deploying daemon mds.g on smithi019 2024-04-25T12:04:50.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:50 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] up:boot 2024-04-25T12:04:50.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:50 smithi019 ceph-mon[29213]: fsmap 9 up:standby 2024-04-25T12:04:50.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:50 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T12:04:51.547 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:04:51 smithi019 systemd[1]: Starting Ceph mds.g for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:04:51.840 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:51 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1063626987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:51.840 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:04:51 smithi019 podman[71370]: 2024-04-25 12:04:51.584875521 +0000 UTC m=+0.018324972 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:04:51.840 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:04:51 smithi019 podman[71370]: 2024-04-25 12:04:51.728304459 +0000 UTC m=+0.161753894 container create ad1bcb2992053629f5bd891450e3be4505182c0d2c2b35e70e4d48dce7f9de1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T12:04:51.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:51 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1063626987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:51.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:51 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1063626987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:52.297 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:04:51 smithi019 podman[71370]: 2024-04-25 12:04:51.981310261 +0000 UTC m=+0.414759694 container init ad1bcb2992053629f5bd891450e3be4505182c0d2c2b35e70e4d48dce7f9de1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-g, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T12:04:52.297 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:04:51 smithi019 podman[71370]: 2024-04-25 12:04:51.984072372 +0000 UTC m=+0.417521814 container start ad1bcb2992053629f5bd891450e3be4505182c0d2c2b35e70e4d48dce7f9de1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T12:04:52.298 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:04:52 smithi019 ceph-mds[71389]: -- 172.21.15.19:0/3909736379 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dcd212fa40 con 0x55dcd2103000 2024-04-25T12:04:52.298 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:04:52 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-g[71385]: starting mds.g at 2024-04-25T12:04:52.298 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:04:52 smithi019 bash[71370]: ad1bcb2992053629f5bd891450e3be4505182c0d2c2b35e70e4d48dce7f9de1a 2024-04-25T12:04:52.298 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:04:52 smithi019 systemd[1]: Started Ceph mds.g for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T12:04:52.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:52 smithi019 ceph-mon[29213]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:52.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:52 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:52.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:52 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:52.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:52 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:52.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:52 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:52.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:52 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:52.905 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:52 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:52.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:52 smithi169 ceph-mon[32710]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:52.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:52 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:52.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:52 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:52.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:52 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:52.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:52 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:52.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:52 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:52.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:52 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:52.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:52 smithi150 ceph-mon[32250]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:52.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:52 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:52.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:52 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:52.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:52 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:52.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:52 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:52.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:52 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:52.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:52 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:52.917 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.026+0000 7f2cacff7640 1 -- 172.21.15.19:0/1048172880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ca8107c70 msgr2=0x7f2ca810a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.026+0000 7f2cacff7640 1 --2- 172.21.15.19:0/1048172880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ca8107c70 0x7f2ca810a0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009ca0 tx=0x7f2ca002a8c0 comp rx=0 tx=0).stop 2024-04-25T12:04:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.026+0000 7f2cacff7640 1 -- 172.21.15.19:0/1048172880 shutdown_connections 2024-04-25T12:04:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.026+0000 7f2cacff7640 1 --2- 172.21.15.19:0/1048172880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ca8107c70 0x7f2ca810a0c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.026+0000 7f2cacff7640 1 --2- 172.21.15.19:0/1048172880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca8105340 0x7f2ca8107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.026+0000 7f2cacff7640 1 --2- 172.21.15.19:0/1048172880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ca8102a10 0x7f2ca8104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.026+0000 7f2cacff7640 1 -- 172.21.15.19:0/1048172880 >> 172.21.15.19:0/1048172880 conn(0x7f2ca80fc400 msgr2=0x7f2ca80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.026+0000 7f2cacff7640 1 -- 172.21.15.19:0/1048172880 shutdown_connections 2024-04-25T12:04:53.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.026+0000 7f2cacff7640 1 -- 172.21.15.19:0/1048172880 wait complete. 2024-04-25T12:04:53.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.027+0000 7f2cacff7640 1 Processor -- start 2024-04-25T12:04:53.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.027+0000 7f2cacff7640 1 -- start start 2024-04-25T12:04:53.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2cacff7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ca8102a10 0x7f2ca81981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:53.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2cacff7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ca8105340 0x7f2ca8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:53.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2cacff7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca8107c70 0x7f2ca819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:53.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2cacff7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca810dc00 con 0x7f2ca8102a10 2024-04-25T12:04:53.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2cacff7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca810da80 con 0x7f2ca8107c70 2024-04-25T12:04:53.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2cacff7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca810dd80 con 0x7f2ca8105340 2024-04-25T12:04:53.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2ca659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ca8102a10 0x7f2ca81981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:53.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2ca659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ca8102a10 0x7f2ca81981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46152/0 (socket says 172.21.15.19:46152) 2024-04-25T12:04:53.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2ca659c640 1 -- 172.21.15.19:0/1950018232 learned_addr learned my addr 172.21.15.19:0/1950018232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:53.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2ca5d9b640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ca8105340 0x7f2ca8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:53.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2ca6d9d640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca8107c70 0x7f2ca819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2ca659c640 1 -- 172.21.15.19:0/1950018232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ca8105340 msgr2=0x7f2ca8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2ca659c640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ca8105340 0x7f2ca8198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2ca659c640 1 -- 172.21.15.19:0/1950018232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca8107c70 msgr2=0x7f2ca819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2ca659c640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca8107c70 0x7f2ca819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2ca659c640 1 -- 172.21.15.19:0/1950018232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca819fe00 con 0x7f2ca8102a10 2024-04-25T12:04:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2ca6d9d640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca8107c70 0x7f2ca819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:04:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2ca659c640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ca8102a10 0x7f2ca81981e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800d950 tx=0x7f2c9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2c977fe640 1 -- 172.21.15.19:0/1950018232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98017070 con 0x7f2ca8102a10 2024-04-25T12:04:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2cacff7640 1 -- 172.21.15.19:0/1950018232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca81a00f0 con 0x7f2ca8102a10 2024-04-25T12:04:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2c977fe640 1 -- 172.21.15.19:0/1950018232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2c98009c30 con 0x7f2ca8102a10 2024-04-25T12:04:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2c977fe640 1 -- 172.21.15.19:0/1950018232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9800f6b0 con 0x7f2ca8102a10 2024-04-25T12:04:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.028+0000 7f2cacff7640 1 -- 172.21.15.19:0/1950018232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca81a0690 con 0x7f2ca8102a10 2024-04-25T12:04:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.030+0000 7f2cacff7640 1 -- 172.21.15.19:0/1950018232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca80fdef0 con 0x7f2ca8102a10 2024-04-25T12:04:53.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.031+0000 7f2c977fe640 1 -- 172.21.15.19:0/1950018232 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2c980109c0 con 0x7f2ca8102a10 2024-04-25T12:04:53.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.031+0000 7f2c977fe640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2c900a3bb0 0x7f2c900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:53.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.031+0000 7f2ca5d9b640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2c900a3bb0 0x7f2c900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:53.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.031+0000 7f2c977fe640 1 -- 172.21.15.19:0/1950018232 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2c980cfc40 con 0x7f2ca8102a10 2024-04-25T12:04:53.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.032+0000 7f2ca5d9b640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2c900a3bb0 0x7f2c900a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c005e10 tx=0x7f2c9c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.036+0000 7f2c977fe640 1 -- 172.21.15.19:0/1950018232 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c98009de0 con 0x7f2ca8102a10 2024-04-25T12:04:53.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.265+0000 7f2cacff7640 1 -- 172.21.15.19:0/1950018232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ca802d5c0 con 0x7f2ca8102a10 2024-04-25T12:04:53.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.266+0000 7f2c977fe640 1 -- 172.21.15.19:0/1950018232 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8842 (secure 0 0 0) 0x7f2c9809cac0 con 0x7f2ca8102a10 2024-04-25T12:04:53.265 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:53.265 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1694897630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1694897630},{"type":"v1","addr":"172.21.15.19:6839","nonce":1694897630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3918801450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3918801450},{"type":"v1","addr":"172.21.15.150:6837","nonce":3918801450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/1723439500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1723439500},{"type":"v1","addr":"172.21.15.169:6839","nonce":1723439500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T12:04:53.265 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 11 2024-04-25T12:04:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.268+0000 7f2cacff7640 1 -- 172.21.15.19:0/1950018232 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2c900a3bb0 msgr2=0x7f2c900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.268+0000 7f2cacff7640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2c900a3bb0 0x7f2c900a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c005e10 tx=0x7f2c9c005da0 comp rx=0 tx=0).stop 2024-04-25T12:04:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.268+0000 7f2cacff7640 1 -- 172.21.15.19:0/1950018232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ca8102a10 msgr2=0x7f2ca81981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.268+0000 7f2cacff7640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ca8102a10 0x7f2ca81981e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800d950 tx=0x7f2c9800de10 comp rx=0 tx=0).stop 2024-04-25T12:04:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.268+0000 7f2cacff7640 1 -- 172.21.15.19:0/1950018232 shutdown_connections 2024-04-25T12:04:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.268+0000 7f2cacff7640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2c900a3bb0 0x7f2c900a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.268+0000 7f2cacff7640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca8107c70 0x7f2ca819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.268+0000 7f2cacff7640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ca8105340 0x7f2ca8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.268+0000 7f2cacff7640 1 --2- 172.21.15.19:0/1950018232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ca8102a10 0x7f2ca81981e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.268+0000 7f2cacff7640 1 -- 172.21.15.19:0/1950018232 >> 172.21.15.19:0/1950018232 conn(0x7f2ca80fc400 msgr2=0x7f2ca8103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.268+0000 7f2cacff7640 1 -- 172.21.15.19:0/1950018232 shutdown_connections 2024-04-25T12:04:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:53.268+0000 7f2cacff7640 1 -- 172.21.15.19:0/1950018232 wait complete. 2024-04-25T12:04:53.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:53 smithi019 ceph-mon[29213]: Deploying daemon mds.a on smithi019 2024-04-25T12:04:53.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:53 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.19:6838/1694897630,v1:172.21.15.19:6839/1694897630] up:boot 2024-04-25T12:04:53.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:53 smithi019 ceph-mon[29213]: fsmap 10 up:standby 2024-04-25T12:04:53.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:53 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T12:04:53.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:53 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1950018232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:53.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:53 smithi169 ceph-mon[32710]: Deploying daemon mds.a on smithi019 2024-04-25T12:04:53.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:53 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.19:6838/1694897630,v1:172.21.15.19:6839/1694897630] up:boot 2024-04-25T12:04:53.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:53 smithi169 ceph-mon[32710]: fsmap 10 up:standby 2024-04-25T12:04:53.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:53 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T12:04:53.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:53 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1950018232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:53.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:53 smithi150 ceph-mon[32250]: Deploying daemon mds.a on smithi019 2024-04-25T12:04:53.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:53 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.19:6838/1694897630,v1:172.21.15.19:6839/1694897630] up:boot 2024-04-25T12:04:53.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:53 smithi150 ceph-mon[32250]: fsmap 10 up:standby 2024-04-25T12:04:53.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:53 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T12:04:53.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:53 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1950018232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:54.547 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:04:54 smithi019 systemd[1]: Starting Ceph mds.a for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:04:54.884 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:54 smithi019 ceph-mon[29213]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:54.886 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:04:54 smithi019 podman[71722]: 2024-04-25 12:04:54.506690902 +0000 UTC m=+0.018558138 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:04:54.886 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:04:54 smithi019 podman[71722]: 2024-04-25 12:04:54.634627029 +0000 UTC m=+0.146494262 container create 765dadd9a116f93efc4f7545e22f589a4b48d600dc137de2d0ea650479dab066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T12:04:54.886 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:04:54 smithi019 podman[71722]: 2024-04-25 12:04:54.887341389 +0000 UTC m=+0.399208622 container init 765dadd9a116f93efc4f7545e22f589a4b48d600dc137de2d0ea650479dab066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T12:04:54.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:54 smithi169 ceph-mon[32710]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:54.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:54 smithi150 ceph-mon[32250]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:55.278 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:55.297 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:04:54 smithi019 podman[71722]: 2024-04-25 12:04:54.890359104 +0000 UTC m=+0.402226338 container start 765dadd9a116f93efc4f7545e22f589a4b48d600dc137de2d0ea650479dab066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-25T12:04:55.297 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:04:54 smithi019 ceph-mds[71741]: -- 172.21.15.19:0/94012137 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562f9cd7ba40 con 0x562f9cd4ec00 2024-04-25T12:04:55.297 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:04:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-a[71737]: starting mds.a at 2024-04-25T12:04:55.297 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:04:54 smithi019 bash[71722]: 765dadd9a116f93efc4f7545e22f589a4b48d600dc137de2d0ea650479dab066 2024-04-25T12:04:55.297 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:04:54 smithi019 systemd[1]: Started Ceph mds.a for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T12:04:55.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.387+0000 7f916b8cb640 1 -- 172.21.15.19:0/815464068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9164102040 msgr2=0x7f91641024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:55.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.387+0000 7f916b8cb640 1 --2- 172.21.15.19:0/815464068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9164102040 0x7f91641024c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9154009900 tx=0x7f915402a410 comp rx=0 tx=0).stop 2024-04-25T12:04:55.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.387+0000 7f916b8cb640 1 -- 172.21.15.19:0/815464068 shutdown_connections 2024-04-25T12:04:55.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.387+0000 7f916b8cb640 1 --2- 172.21.15.19:0/815464068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9164102a00 0x7f9164109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:55.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.387+0000 7f916b8cb640 1 --2- 172.21.15.19:0/815464068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9164102040 0x7f91641024c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:55.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.387+0000 7f916b8cb640 1 --2- 172.21.15.19:0/815464068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9164100e60 0x7f9164101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:55.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.387+0000 7f916b8cb640 1 -- 172.21.15.19:0/815464068 >> 172.21.15.19:0/815464068 conn(0x7f91640fc440 msgr2=0x7f91640fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.387+0000 7f916b8cb640 1 -- 172.21.15.19:0/815464068 shutdown_connections 2024-04-25T12:04:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.387+0000 7f916b8cb640 1 -- 172.21.15.19:0/815464068 wait complete. 2024-04-25T12:04:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.388+0000 7f916b8cb640 1 Processor -- start 2024-04-25T12:04:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.388+0000 7f916b8cb640 1 -- start start 2024-04-25T12:04:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.388+0000 7f916b8cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9164100e60 0x7f916406b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.388+0000 7f916b8cb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9164102040 0x7f916419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.388+0000 7f916b8cb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9164102a00 0x7f91641a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.388+0000 7f916b8cb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f916410cb50 con 0x7f9164100e60 2024-04-25T12:04:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.388+0000 7f916b8cb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f916410c9d0 con 0x7f9164102a00 2024-04-25T12:04:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.388+0000 7f916b8cb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f916410ccd0 con 0x7f9164102040 2024-04-25T12:04:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.388+0000 7f9169667640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9164100e60 0x7f916406b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.388+0000 7f9169667640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9164100e60 0x7f916406b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46200/0 (socket says 172.21.15.19:46200) 2024-04-25T12:04:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.388+0000 7f9169667640 1 -- 172.21.15.19:0/560789535 learned_addr learned my addr 172.21.15.19:0/560789535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.388+0000 7f9168e66640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9164102040 0x7f916419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.388+0000 7f9169e68640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9164102a00 0x7f91641a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.389+0000 7f9169667640 1 -- 172.21.15.19:0/560789535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9164102040 msgr2=0x7f916419f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.389+0000 7f9169667640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9164102040 0x7f916419f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.389+0000 7f9169667640 1 -- 172.21.15.19:0/560789535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9164102a00 msgr2=0x7f91641a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.389+0000 7f9169667640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9164102a00 0x7f91641a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.389+0000 7f9169667640 1 -- 172.21.15.19:0/560789535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91641a2130 con 0x7f9164100e60 2024-04-25T12:04:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.389+0000 7f9169e68640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9164102a00 0x7f91641a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:04:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.389+0000 7f9169667640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9164100e60 0x7f916406b180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f916000d950 tx=0x7f916000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.389+0000 7f915a7fc640 1 -- 172.21.15.19:0/560789535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9160017070 con 0x7f9164100e60 2024-04-25T12:04:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.389+0000 7f915a7fc640 1 -- 172.21.15.19:0/560789535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9160009c30 con 0x7f9164100e60 2024-04-25T12:04:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.389+0000 7f916b8cb640 1 -- 172.21.15.19:0/560789535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91641a2420 con 0x7f9164100e60 2024-04-25T12:04:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.389+0000 7f915a7fc640 1 -- 172.21.15.19:0/560789535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916000f620 con 0x7f9164100e60 2024-04-25T12:04:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.389+0000 7f916b8cb640 1 -- 172.21.15.19:0/560789535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91641a28f0 con 0x7f9164100e60 2024-04-25T12:04:55.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.390+0000 7f916b8cb640 1 -- 172.21.15.19:0/560789535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91640691c0 con 0x7f9164100e60 2024-04-25T12:04:55.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.391+0000 7f915a7fc640 1 -- 172.21.15.19:0/560789535 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f916001a070 con 0x7f9164100e60 2024-04-25T12:04:55.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.392+0000 7f915a7fc640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f91500a3bb0 0x7f91500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:55.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.392+0000 7f915a7fc640 1 -- 172.21.15.19:0/560789535 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f91600cfc60 con 0x7f9164100e60 2024-04-25T12:04:55.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.392+0000 7f9168e66640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f91500a3bb0 0x7f91500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:55.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.392+0000 7f9168e66640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f91500a3bb0 0x7f91500a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9154000c00 tx=0x7f9154031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:55.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.396+0000 7f915a7fc640 1 -- 172.21.15.19:0/560789535 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9160009de0 con 0x7f9164100e60 2024-04-25T12:04:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.622+0000 7f916b8cb640 1 -- 172.21.15.19:0/560789535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9164108200 con 0x7f9164100e60 2024-04-25T12:04:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.623+0000 7f915a7fc640 1 -- 172.21.15.19:0/560789535 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7f916009ca40 con 0x7f9164100e60 2024-04-25T12:04:55.620 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:55.620 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1694897630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1694897630},{"type":"v1","addr":"172.21.15.19:6839","nonce":1694897630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/3247157172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3247157172},{"type":"v1","addr":"172.21.15.19:6841","nonce":3247157172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3918801450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3918801450},{"type":"v1","addr":"172.21.15.150:6837","nonce":3918801450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/1723439500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1723439500},{"type":"v1","addr":"172.21.15.169:6839","nonce":1723439500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T12:04:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 12 2024-04-25T12:04:55.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.625+0000 7f916b8cb640 1 -- 172.21.15.19:0/560789535 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f91500a3bb0 msgr2=0x7f91500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.625+0000 7f916b8cb640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f91500a3bb0 0x7f91500a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9154000c00 tx=0x7f9154031510 comp rx=0 tx=0).stop 2024-04-25T12:04:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.625+0000 7f916b8cb640 1 -- 172.21.15.19:0/560789535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9164100e60 msgr2=0x7f916406b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.625+0000 7f916b8cb640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9164100e60 0x7f916406b180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f916000d950 tx=0x7f916000de10 comp rx=0 tx=0).stop 2024-04-25T12:04:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.625+0000 7f916b8cb640 1 -- 172.21.15.19:0/560789535 shutdown_connections 2024-04-25T12:04:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.625+0000 7f916b8cb640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f91500a3bb0 0x7f91500a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.625+0000 7f916b8cb640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9164102a00 0x7f91641a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.625+0000 7f916b8cb640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9164102040 0x7f916419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.625+0000 7f916b8cb640 1 --2- 172.21.15.19:0/560789535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9164100e60 0x7f916406b180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.625+0000 7f916b8cb640 1 -- 172.21.15.19:0/560789535 >> 172.21.15.19:0/560789535 conn(0x7f91640fc440 msgr2=0x7f91641053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.625+0000 7f916b8cb640 1 -- 172.21.15.19:0/560789535 shutdown_connections 2024-04-25T12:04:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:55.625+0000 7f916b8cb640 1 -- 172.21.15.19:0/560789535 wait complete. 2024-04-25T12:04:56.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:56 smithi019 ceph-mon[29213]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:56.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:56.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:56.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:56.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:56.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:56.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:56.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:56 smithi019 ceph-mon[29213]: Deploying daemon mds.b on smithi150 2024-04-25T12:04:56.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:56 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] up:boot 2024-04-25T12:04:56.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:56 smithi019 ceph-mon[29213]: fsmap 11 up:standby 2024-04-25T12:04:56.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:56 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T12:04:56.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:56 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/560789535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:56 smithi169 ceph-mon[32710]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:56.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:56 smithi169 ceph-mon[32710]: Deploying daemon mds.b on smithi150 2024-04-25T12:04:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:56 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] up:boot 2024-04-25T12:04:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:56 smithi169 ceph-mon[32710]: fsmap 11 up:standby 2024-04-25T12:04:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:56 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T12:04:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:56 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/560789535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:56.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:56 smithi150 ceph-mon[32250]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:56.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:56.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:56.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:56.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T12:04:56.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T12:04:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:04:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:56 smithi150 ceph-mon[32250]: Deploying daemon mds.b on smithi150 2024-04-25T12:04:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:56 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] up:boot 2024-04-25T12:04:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:56 smithi150 ceph-mon[32250]: fsmap 11 up:standby 2024-04-25T12:04:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:56 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T12:04:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:56 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/560789535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:57.634 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:04:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.752+0000 7f375b1f7640 1 -- 172.21.15.19:0/3072271666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3754103190 msgr2=0x7f3754103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.752+0000 7f375b1f7640 1 --2- 172.21.15.19:0/3072271666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3754103190 0x7f3754103610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3744009900 tx=0x7f374402a410 comp rx=0 tx=0).stop 2024-04-25T12:04:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 -- 172.21.15.19:0/3072271666 shutdown_connections 2024-04-25T12:04:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 --2- 172.21.15.19:0/3072271666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3754103b50 0x7f375410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 --2- 172.21.15.19:0/3072271666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3754103190 0x7f3754103610 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 --2- 172.21.15.19:0/3072271666 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3754101fa0 0x7f37541023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 -- 172.21.15.19:0/3072271666 >> 172.21.15.19:0/3072271666 conn(0x7f37540fd530 msgr2=0x7f37540ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 -- 172.21.15.19:0/3072271666 shutdown_connections 2024-04-25T12:04:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 -- 172.21.15.19:0/3072271666 wait complete. 2024-04-25T12:04:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 Processor -- start 2024-04-25T12:04:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 -- start start 2024-04-25T12:04:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3754101fa0 0x7f3754198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3754103190 0x7f3754198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3754103b50 0x7f375419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f375410dc00 con 0x7f3754101fa0 2024-04-25T12:04:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f375410da80 con 0x7f3754103190 2024-04-25T12:04:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.753+0000 7f375b1f7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f375410dd80 con 0x7f3754103b50 2024-04-25T12:04:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f3758f93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3754101fa0 0x7f3754198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f3758f93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3754101fa0 0x7f3754198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46214/0 (socket says 172.21.15.19:46214) 2024-04-25T12:04:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f3758f93640 1 -- 172.21.15.19:0/3847830168 learned_addr learned my addr 172.21.15.19:0/3847830168 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:04:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f374bfff640 1 --2- 172.21.15.19:0/3847830168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3754103190 0x7f3754198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f3759794640 1 --2- 172.21.15.19:0/3847830168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3754103b50 0x7f375419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f3758f93640 1 -- 172.21.15.19:0/3847830168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3754103b50 msgr2=0x7f375419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f3758f93640 1 --2- 172.21.15.19:0/3847830168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3754103b50 0x7f375419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f3758f93640 1 -- 172.21.15.19:0/3847830168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3754103190 msgr2=0x7f3754198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f3758f93640 1 --2- 172.21.15.19:0/3847830168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3754103190 0x7f3754198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f3758f93640 1 -- 172.21.15.19:0/3847830168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375419fe70 con 0x7f3754101fa0 2024-04-25T12:04:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f3758f93640 1 --2- 172.21.15.19:0/3847830168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3754101fa0 0x7f3754198250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f375000c970 tx=0x7f375000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f3749ffb640 1 -- 172.21.15.19:0/3847830168 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750004220 con 0x7f3754101fa0 2024-04-25T12:04:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f3749ffb640 1 -- 172.21.15.19:0/3847830168 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f37500043c0 con 0x7f3754101fa0 2024-04-25T12:04:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f375b1f7640 1 -- 172.21.15.19:0/3847830168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37541a0160 con 0x7f3754101fa0 2024-04-25T12:04:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f3749ffb640 1 -- 172.21.15.19:0/3847830168 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750004a00 con 0x7f3754101fa0 2024-04-25T12:04:57.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.754+0000 7f375b1f7640 1 -- 172.21.15.19:0/3847830168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37541a0680 con 0x7f3754101fa0 2024-04-25T12:04:57.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.755+0000 7f375b1f7640 1 -- 172.21.15.19:0/3847830168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37541092c0 con 0x7f3754101fa0 2024-04-25T12:04:57.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.756+0000 7f3749ffb640 1 -- 172.21.15.19:0/3847830168 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f37500079b0 con 0x7f3754101fa0 2024-04-25T12:04:57.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.757+0000 7f3749ffb640 1 --2- 172.21.15.19:0/3847830168 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f373c0a3c30 0x7f373c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:04:57.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.757+0000 7f3749ffb640 1 -- 172.21.15.19:0/3847830168 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f37500cfcd0 con 0x7f3754101fa0 2024-04-25T12:04:57.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.757+0000 7f374bfff640 1 --2- 172.21.15.19:0/3847830168 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f373c0a3c30 0x7f373c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:04:57.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.758+0000 7f374bfff640 1 --2- 172.21.15.19:0/3847830168 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f373c0a3c30 0x7f373c0a60f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3744000c00 tx=0x7f3744031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:04:57.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.762+0000 7f3749ffb640 1 -- 172.21.15.19:0/3847830168 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3750014030 con 0x7f3754101fa0 2024-04-25T12:04:57.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.991+0000 7f375b1f7640 1 -- 172.21.15.19:0/3847830168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f375402d5c0 con 0x7f3754101fa0 2024-04-25T12:04:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.992+0000 7f3749ffb640 1 -- 172.21.15.19:0/3847830168 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7f375009cb50 con 0x7f3754101fa0 2024-04-25T12:04:57.989 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:04:57.989 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1694897630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1694897630},{"type":"v1","addr":"172.21.15.19:6839","nonce":1694897630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/3247157172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3247157172},{"type":"v1","addr":"172.21.15.19:6841","nonce":3247157172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3918801450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3918801450},{"type":"v1","addr":"172.21.15.150:6837","nonce":3918801450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/1723439500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1723439500},{"type":"v1","addr":"172.21.15.169:6839","nonce":1723439500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T12:04:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 12 2024-04-25T12:04:57.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.994+0000 7f375b1f7640 1 -- 172.21.15.19:0/3847830168 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f373c0a3c30 msgr2=0x7f373c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.994+0000 7f375b1f7640 1 --2- 172.21.15.19:0/3847830168 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f373c0a3c30 0x7f373c0a60f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3744000c00 tx=0x7f3744031510 comp rx=0 tx=0).stop 2024-04-25T12:04:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.994+0000 7f375b1f7640 1 -- 172.21.15.19:0/3847830168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3754101fa0 msgr2=0x7f3754198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:04:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.994+0000 7f375b1f7640 1 --2- 172.21.15.19:0/3847830168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3754101fa0 0x7f3754198250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f375000c970 tx=0x7f375000ce30 comp rx=0 tx=0).stop 2024-04-25T12:04:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.994+0000 7f375b1f7640 1 -- 172.21.15.19:0/3847830168 shutdown_connections 2024-04-25T12:04:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.994+0000 7f375b1f7640 1 --2- 172.21.15.19:0/3847830168 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f373c0a3c30 0x7f373c0a60f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.994+0000 7f375b1f7640 1 --2- 172.21.15.19:0/3847830168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3754103b50 0x7f375419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.994+0000 7f375b1f7640 1 --2- 172.21.15.19:0/3847830168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3754103190 0x7f3754198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.994+0000 7f375b1f7640 1 --2- 172.21.15.19:0/3847830168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3754101fa0 0x7f3754198250 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:04:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.994+0000 7f375b1f7640 1 -- 172.21.15.19:0/3847830168 >> 172.21.15.19:0/3847830168 conn(0x7f37540fd530 msgr2=0x7f3754106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:04:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.994+0000 7f375b1f7640 1 -- 172.21.15.19:0/3847830168 shutdown_connections 2024-04-25T12:04:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:04:57.994+0000 7f375b1f7640 1 -- 172.21.15.19:0/3847830168 wait complete. 2024-04-25T12:04:58.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:58 smithi150 ceph-mon[32250]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:58.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:58 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3847830168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:58.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:58.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:58 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:58.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:58 smithi019 ceph-mon[29213]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:58.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:58 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3847830168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:58.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:58.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:58 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:58.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:58 smithi169 ceph-mon[32710]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:04:58.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:58 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3847830168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:04:58.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:58.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:58 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:59.313 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:59 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] up:boot 2024-04-25T12:04:59.313 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:59 smithi019 ceph-mon[29213]: fsmap 12 up:standby 2024-04-25T12:04:59.313 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:59 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T12:04:59.313 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:59 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:59.313 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:59 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:59.314 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:59 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:59.314 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:04:59 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:04:59.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:59 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] up:boot 2024-04-25T12:04:59.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:59 smithi169 ceph-mon[32710]: fsmap 12 up:standby 2024-04-25T12:04:59.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:59 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T12:04:59.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:59 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:59.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:59 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:59.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:59 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:59.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:04:59 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:04:59.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:59 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] up:boot 2024-04-25T12:04:59.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:59 smithi150 ceph-mon[32250]: fsmap 12 up:standby 2024-04-25T12:04:59.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:59 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T12:04:59.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:59 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:59.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:59 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:59.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:59 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:04:59.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:04:59 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:05:00.004 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:05:00.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.119+0000 7f3149acf640 1 -- 172.21.15.19:0/3634724239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3144100e90 msgr2=0x7f31441012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.119+0000 7f3149acf640 1 --2- 172.21.15.19:0/3634724239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3144100e90 0x7f31441012b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3134009900 tx=0x7f313402a410 comp rx=0 tx=0).stop 2024-04-25T12:05:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.119+0000 7f3149acf640 1 -- 172.21.15.19:0/3634724239 shutdown_connections 2024-04-25T12:05:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.119+0000 7f3149acf640 1 --2- 172.21.15.19:0/3634724239 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f31441029a0 0x7f31441092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.119+0000 7f3149acf640 1 --2- 172.21.15.19:0/3634724239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3144101fe0 0x7f3144102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.119+0000 7f3149acf640 1 --2- 172.21.15.19:0/3634724239 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3144100e90 0x7f31441012b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.119+0000 7f3149acf640 1 -- 172.21.15.19:0/3634724239 >> 172.21.15.19:0/3634724239 conn(0x7f31440fc400 msgr2=0x7f31440fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.119+0000 7f3149acf640 1 -- 172.21.15.19:0/3634724239 shutdown_connections 2024-04-25T12:05:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.119+0000 7f3149acf640 1 -- 172.21.15.19:0/3634724239 wait complete. 2024-04-25T12:05:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.120+0000 7f3149acf640 1 Processor -- start 2024-04-25T12:05:00.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.120+0000 7f3149acf640 1 -- start start 2024-04-25T12:05:00.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.120+0000 7f3149acf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3144100e90 0x7f3144069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:00.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.120+0000 7f3149acf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3144101fe0 0x7f314419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:00.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.120+0000 7f3149acf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31441029a0 0x7f314419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:00.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.120+0000 7f3149acf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f314410cb20 con 0x7f31441029a0 2024-04-25T12:05:00.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.120+0000 7f3149acf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f314410c9a0 con 0x7f3144101fe0 2024-04-25T12:05:00.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.120+0000 7f3149acf640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f314410cca0 con 0x7f3144100e90 2024-04-25T12:05:00.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.120+0000 7f3143fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31441029a0 0x7f314419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:00.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.120+0000 7f3143fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31441029a0 0x7f314419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46242/0 (socket says 172.21.15.19:46242) 2024-04-25T12:05:00.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.120+0000 7f3143fff640 1 -- 172.21.15.19:0/785146057 learned_addr learned my addr 172.21.15.19:0/785146057 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:00.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.120+0000 7f31437fe640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3144100e90 0x7f3144069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:00.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.120+0000 7f313affd640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3144101fe0 0x7f314419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.121+0000 7f3143fff640 1 -- 172.21.15.19:0/785146057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3144100e90 msgr2=0x7f3144069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.121+0000 7f3143fff640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3144100e90 0x7f3144069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.121+0000 7f3143fff640 1 -- 172.21.15.19:0/785146057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3144101fe0 msgr2=0x7f314419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.121+0000 7f3143fff640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3144101fe0 0x7f314419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.121+0000 7f3143fff640 1 -- 172.21.15.19:0/785146057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314419fee0 con 0x7f31441029a0 2024-04-25T12:05:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.121+0000 7f313affd640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3144101fe0 0x7f314419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:05:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.121+0000 7f3143fff640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31441029a0 0x7f314419f620 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f313c00cad0 tx=0x7f313c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.121+0000 7f31417fa640 1 -- 172.21.15.19:0/785146057 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c00ae20 con 0x7f31441029a0 2024-04-25T12:05:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.121+0000 7f31417fa640 1 -- 172.21.15.19:0/785146057 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f313c005020 con 0x7f31441029a0 2024-04-25T12:05:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.121+0000 7f3149acf640 1 -- 172.21.15.19:0/785146057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31441a01d0 con 0x7f31441029a0 2024-04-25T12:05:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.121+0000 7f31417fa640 1 -- 172.21.15.19:0/785146057 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c00f6b0 con 0x7f31441029a0 2024-04-25T12:05:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.121+0000 7f3149acf640 1 -- 172.21.15.19:0/785146057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31441a0720 con 0x7f31441029a0 2024-04-25T12:05:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.122+0000 7f3149acf640 1 -- 172.21.15.19:0/785146057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31441081a0 con 0x7f31441029a0 2024-04-25T12:05:00.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.123+0000 7f31417fa640 1 -- 172.21.15.19:0/785146057 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f313c0051d0 con 0x7f31441029a0 2024-04-25T12:05:00.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.124+0000 7f31417fa640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31300a3bb0 0x7f31300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:00.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.124+0000 7f31417fa640 1 -- 172.21.15.19:0/785146057 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f313c0d03b0 con 0x7f31441029a0 2024-04-25T12:05:00.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.124+0000 7f31437fe640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31300a3bb0 0x7f31300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:00.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.125+0000 7f31437fe640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31300a3bb0 0x7f31300a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3134000c00 tx=0x7f3134031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.128+0000 7f31417fa640 1 -- 172.21.15.19:0/785146057 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f313c014030 con 0x7f31441029a0 2024-04-25T12:05:00.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:00 smithi019 ceph-mon[29213]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:05:00.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:00 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.356+0000 7f3149acf640 1 -- 172.21.15.19:0/785146057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f314402d5c0 con 0x7f31441029a0 2024-04-25T12:05:00.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.357+0000 7f31417fa640 1 -- 172.21.15.19:0/785146057 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10497 (secure 0 0 0) 0x7f313c09d1d0 con 0x7f31441029a0 2024-04-25T12:05:00.355 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:05:00.355 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1694897630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1694897630},{"type":"v1","addr":"172.21.15.19:6839","nonce":1694897630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/3247157172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3247157172},{"type":"v1","addr":"172.21.15.19:6841","nonce":3247157172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3918801450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3918801450},{"type":"v1","addr":"172.21.15.150:6837","nonce":3918801450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24500,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2553528303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2553528303},{"type":"v1","addr":"172.21.15.150:6839","nonce":2553528303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6839/1723439500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1723439500},{"type":"v1","addr":"172.21.15.169:6839","nonce":1723439500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T12:05:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 13 2024-04-25T12:05:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.359+0000 7f3149acf640 1 -- 172.21.15.19:0/785146057 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31300a3bb0 msgr2=0x7f31300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.359+0000 7f3149acf640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31300a3bb0 0x7f31300a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3134000c00 tx=0x7f3134031510 comp rx=0 tx=0).stop 2024-04-25T12:05:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.359+0000 7f3149acf640 1 -- 172.21.15.19:0/785146057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31441029a0 msgr2=0x7f314419f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.359+0000 7f3149acf640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31441029a0 0x7f314419f620 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f313c00cad0 tx=0x7f313c00cf90 comp rx=0 tx=0).stop 2024-04-25T12:05:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.359+0000 7f3149acf640 1 -- 172.21.15.19:0/785146057 shutdown_connections 2024-04-25T12:05:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.359+0000 7f3149acf640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f31300a3bb0 0x7f31300a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.359+0000 7f3149acf640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31441029a0 0x7f314419f620 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.359+0000 7f3149acf640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3144101fe0 0x7f314419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.359+0000 7f3149acf640 1 --2- 172.21.15.19:0/785146057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3144100e90 0x7f3144069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.359+0000 7f3149acf640 1 -- 172.21.15.19:0/785146057 >> 172.21.15.19:0/785146057 conn(0x7f31440fc400 msgr2=0x7f3144105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.359+0000 7f3149acf640 1 -- 172.21.15.19:0/785146057 shutdown_connections 2024-04-25T12:05:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.359+0000 7f3149acf640 1 -- 172.21.15.19:0/785146057 wait complete. 2024-04-25T12:05:00.368 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:05:00.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:00 smithi169 ceph-mon[32710]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:05:00.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:00 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.412 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:00 smithi150 ceph-mon[32250]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:05:00.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:00 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.479+0000 7fe8250cf640 1 -- 172.21.15.19:0/3271630656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820101fe0 msgr2=0x7fe820102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.479+0000 7fe8250cf640 1 --2- 172.21.15.19:0/3271630656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820101fe0 0x7fe820102460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe814009900 tx=0x7fe81402a410 comp rx=0 tx=0).stop 2024-04-25T12:05:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.479+0000 7fe8250cf640 1 -- 172.21.15.19:0/3271630656 shutdown_connections 2024-04-25T12:05:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.479+0000 7fe8250cf640 1 --2- 172.21.15.19:0/3271630656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8201029a0 0x7fe8201092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.479+0000 7fe8250cf640 1 --2- 172.21.15.19:0/3271630656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820101fe0 0x7fe820102460 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.479+0000 7fe8250cf640 1 --2- 172.21.15.19:0/3271630656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe820100e90 0x7fe8201012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.479+0000 7fe8250cf640 1 -- 172.21.15.19:0/3271630656 >> 172.21.15.19:0/3271630656 conn(0x7fe8200fc400 msgr2=0x7fe8200fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.479+0000 7fe8250cf640 1 -- 172.21.15.19:0/3271630656 shutdown_connections 2024-04-25T12:05:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.479+0000 7fe8250cf640 1 -- 172.21.15.19:0/3271630656 wait complete. 2024-04-25T12:05:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe8250cf640 1 Processor -- start 2024-04-25T12:05:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe8250cf640 1 -- start start 2024-04-25T12:05:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe8250cf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe820100e90 0x7fe8201982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe8250cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820101fe0 0x7fe820198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe8250cf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8201029a0 0x7fe82019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe8250cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe82010cb20 con 0x7fe820101fe0 2024-04-25T12:05:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe8250cf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe82010c9a0 con 0x7fe8201029a0 2024-04-25T12:05:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe8250cf640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe82010cca0 con 0x7fe820100e90 2024-04-25T12:05:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe81e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820101fe0 0x7fe820198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe81e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820101fe0 0x7fe820198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46252/0 (socket says 172.21.15.19:46252) 2024-04-25T12:05:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe81e59c640 1 -- 172.21.15.19:0/1327938119 learned_addr learned my addr 172.21.15.19:0/1327938119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe81ed9d640 1 --2- 172.21.15.19:0/1327938119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe820100e90 0x7fe8201982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe81f59e640 1 --2- 172.21.15.19:0/1327938119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8201029a0 0x7fe82019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe81e59c640 1 -- 172.21.15.19:0/1327938119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe820100e90 msgr2=0x7fe8201982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe81e59c640 1 --2- 172.21.15.19:0/1327938119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe820100e90 0x7fe8201982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe81e59c640 1 -- 172.21.15.19:0/1327938119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8201029a0 msgr2=0x7fe82019f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe81e59c640 1 --2- 172.21.15.19:0/1327938119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8201029a0 0x7fe82019f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.480+0000 7fe81e59c640 1 -- 172.21.15.19:0/1327938119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82019ff00 con 0x7fe820101fe0 2024-04-25T12:05:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.481+0000 7fe81e59c640 1 --2- 172.21.15.19:0/1327938119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820101fe0 0x7fe820198820 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe814006010 tx=0x7fe8140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.481+0000 7fe807fff640 1 -- 172.21.15.19:0/1327938119 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81403f030 con 0x7fe820101fe0 2024-04-25T12:05:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.481+0000 7fe8250cf640 1 -- 172.21.15.19:0/1327938119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8201a0190 con 0x7fe820101fe0 2024-04-25T12:05:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.481+0000 7fe807fff640 1 -- 172.21.15.19:0/1327938119 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe814003cd0 con 0x7fe820101fe0 2024-04-25T12:05:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.481+0000 7fe807fff640 1 -- 172.21.15.19:0/1327938119 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814035070 con 0x7fe820101fe0 2024-04-25T12:05:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.481+0000 7fe8250cf640 1 -- 172.21.15.19:0/1327938119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8201a06d0 con 0x7fe820101fe0 2024-04-25T12:05:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.482+0000 7fe8250cf640 1 -- 172.21.15.19:0/1327938119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8201a0930 con 0x7fe820101fe0 2024-04-25T12:05:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.483+0000 7fe807fff640 1 -- 172.21.15.19:0/1327938119 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe814042030 con 0x7fe820101fe0 2024-04-25T12:05:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.484+0000 7fe807fff640 1 --2- 172.21.15.19:0/1327938119 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe8080a3bb0 0x7fe8080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.484+0000 7fe807fff640 1 -- 172.21.15.19:0/1327938119 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe8140f0980 con 0x7fe820101fe0 2024-04-25T12:05:00.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.484+0000 7fe81ed9d640 1 --2- 172.21.15.19:0/1327938119 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe8080a3bb0 0x7fe8080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:00.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.484+0000 7fe81ed9d640 1 --2- 172.21.15.19:0/1327938119 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe8080a3bb0 0x7fe8080a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe810009bd0 tx=0x7fe810005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:00.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.488+0000 7fe807fff640 1 -- 172.21.15.19:0/1327938119 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8140bd760 con 0x7fe820101fe0 2024-04-25T12:05:00.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.680+0000 7fe8250cf640 1 -- 172.21.15.19:0/1327938119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8201a1000 con 0x7fe820101fe0 2024-04-25T12:05:00.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.681+0000 7fe807fff640 1 -- 172.21.15.19:0/1327938119 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18440 (secure 0 0 0) 0x7fe8140bd060 con 0x7fe820101fe0 2024-04-25T12:05:00.679 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:05:00.679 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":81,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:04:14.922729+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:05:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.683+0000 7fe8250cf640 1 -- 172.21.15.19:0/1327938119 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe8080a3bb0 msgr2=0x7fe8080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.683+0000 7fe8250cf640 1 --2- 172.21.15.19:0/1327938119 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe8080a3bb0 0x7fe8080a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe810009bd0 tx=0x7fe810005f90 comp rx=0 tx=0).stop 2024-04-25T12:05:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.683+0000 7fe8250cf640 1 -- 172.21.15.19:0/1327938119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820101fe0 msgr2=0x7fe820198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.683+0000 7fe8250cf640 1 --2- 172.21.15.19:0/1327938119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820101fe0 0x7fe820198820 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe814006010 tx=0x7fe8140042f0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.683+0000 7fe8250cf640 1 -- 172.21.15.19:0/1327938119 shutdown_connections 2024-04-25T12:05:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.683+0000 7fe8250cf640 1 --2- 172.21.15.19:0/1327938119 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe8080a3bb0 0x7fe8080a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.683+0000 7fe8250cf640 1 --2- 172.21.15.19:0/1327938119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8201029a0 0x7fe82019f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.683+0000 7fe8250cf640 1 --2- 172.21.15.19:0/1327938119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe820101fe0 0x7fe820198820 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.683+0000 7fe8250cf640 1 --2- 172.21.15.19:0/1327938119 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe820100e90 0x7fe8201982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.683+0000 7fe8250cf640 1 -- 172.21.15.19:0/1327938119 >> 172.21.15.19:0/1327938119 conn(0x7fe8200fc400 msgr2=0x7fe8200690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:00.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.683+0000 7fe8250cf640 1 -- 172.21.15.19:0/1327938119 shutdown_connections 2024-04-25T12:05:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.683+0000 7fe8250cf640 1 -- 172.21.15.19:0/1327938119 wait complete. 2024-04-25T12:05:00.692 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:00:20.898809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T12:05:00.692 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:05:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.800+0000 7ff6af708640 1 -- 172.21.15.19:0/3128041613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a8103300 msgr2=0x7ff6a8103780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.800+0000 7ff6af708640 1 --2- 172.21.15.19:0/3128041613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a8103300 0x7ff6a8103780 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff698009930 tx=0x7ff69802a420 comp rx=0 tx=0).stop 2024-04-25T12:05:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.800+0000 7ff6af708640 1 -- 172.21.15.19:0/3128041613 shutdown_connections 2024-04-25T12:05:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.800+0000 7ff6af708640 1 --2- 172.21.15.19:0/3128041613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6a8103cc0 0x7ff6a810a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.800+0000 7ff6af708640 1 --2- 172.21.15.19:0/3128041613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a8103300 0x7ff6a8103780 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.800+0000 7ff6af708640 1 --2- 172.21.15.19:0/3128041613 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6a8100290 0x7ff6a81006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.800+0000 7ff6af708640 1 -- 172.21.15.19:0/3128041613 >> 172.21.15.19:0/3128041613 conn(0x7ff6a80fbb80 msgr2=0x7ff6a80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.800+0000 7ff6af708640 1 -- 172.21.15.19:0/3128041613 shutdown_connections 2024-04-25T12:05:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.800+0000 7ff6af708640 1 -- 172.21.15.19:0/3128041613 wait complete. 2024-04-25T12:05:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6af708640 1 Processor -- start 2024-04-25T12:05:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6af708640 1 -- start start 2024-04-25T12:05:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6af708640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6a8100290 0x7ff6a819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6af708640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a8103300 0x7ff6a819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6af708640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6a8103cc0 0x7ff6a81a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6af708640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a810dca0 con 0x7ff6a8103300 2024-04-25T12:05:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6af708640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a810db20 con 0x7ff6a8100290 2024-04-25T12:05:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6af708640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a810de20 con 0x7ff6a8103cc0 2024-04-25T12:05:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6acca3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a8103300 0x7ff6a819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6acca3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a8103300 0x7ff6a819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46272/0 (socket says 172.21.15.19:46272) 2024-04-25T12:05:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6acca3640 1 -- 172.21.15.19:0/1113852295 learned_addr learned my addr 172.21.15.19:0/1113852295 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6ad4a4640 1 --2- 172.21.15.19:0/1113852295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6a8100290 0x7ff6a819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6adca5640 1 --2- 172.21.15.19:0/1113852295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6a8103cc0 0x7ff6a81a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6acca3640 1 -- 172.21.15.19:0/1113852295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6a8103cc0 msgr2=0x7ff6a81a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6acca3640 1 --2- 172.21.15.19:0/1113852295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6a8103cc0 0x7ff6a81a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6acca3640 1 -- 172.21.15.19:0/1113852295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6a8100290 msgr2=0x7ff6a819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6acca3640 1 --2- 172.21.15.19:0/1113852295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6a8100290 0x7ff6a819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6acca3640 1 -- 172.21.15.19:0/1113852295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a81a21a0 con 0x7ff6a8103300 2024-04-25T12:05:00.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.801+0000 7ff6acca3640 1 --2- 172.21.15.19:0/1113852295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a8103300 0x7ff6a819aac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff698009a50 tx=0x7ff6980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.802+0000 7ff69e7fc640 1 -- 172.21.15.19:0/1113852295 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69803f030 con 0x7ff6a8103300 2024-04-25T12:05:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.802+0000 7ff6af708640 1 -- 172.21.15.19:0/1113852295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a81a2430 con 0x7ff6a8103300 2024-04-25T12:05:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.802+0000 7ff69e7fc640 1 -- 172.21.15.19:0/1113852295 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff698003cd0 con 0x7ff6a8103300 2024-04-25T12:05:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.802+0000 7ff69e7fc640 1 -- 172.21.15.19:0/1113852295 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698035070 con 0x7ff6a8103300 2024-04-25T12:05:00.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.802+0000 7ff6af708640 1 -- 172.21.15.19:0/1113852295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a81a29f0 con 0x7ff6a8103300 2024-04-25T12:05:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.803+0000 7ff6af708640 1 -- 172.21.15.19:0/1113852295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a81a2c50 con 0x7ff6a8103300 2024-04-25T12:05:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.804+0000 7ff69e7fc640 1 -- 172.21.15.19:0/1113852295 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff698042030 con 0x7ff6a8103300 2024-04-25T12:05:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.805+0000 7ff69e7fc640 1 --2- 172.21.15.19:0/1113852295 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff6940a3bb0 0x7ff6940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.805+0000 7ff6ad4a4640 1 --2- 172.21.15.19:0/1113852295 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff6940a3bb0 0x7ff6940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.805+0000 7ff69e7fc640 1 -- 172.21.15.19:0/1113852295 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff6980f09c0 con 0x7ff6a8103300 2024-04-25T12:05:00.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.805+0000 7ff6ad4a4640 1 --2- 172.21.15.19:0/1113852295 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff6940a3bb0 0x7ff6940a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4005fd0 tx=0x7ff6a400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:00.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.808+0000 7ff69e7fc640 1 -- 172.21.15.19:0/1113852295 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6980bd1f0 con 0x7ff6a8103300 2024-04-25T12:05:00.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.998+0000 7ff6af708640 1 -- 172.21.15.19:0/1113852295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff6a81a2fe0 con 0x7ff6a8103300 2024-04-25T12:05:00.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:00.999+0000 7ff69e7fc640 1 -- 172.21.15.19:0/1113852295 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7ff6980c14f0 con 0x7ff6a8103300 2024-04-25T12:05:00.996 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T12:05:00.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.001+0000 7ff6af708640 1 -- 172.21.15.19:0/1113852295 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff6940a3bb0 msgr2=0x7ff6940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.001+0000 7ff6af708640 1 --2- 172.21.15.19:0/1113852295 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff6940a3bb0 0x7ff6940a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4005fd0 tx=0x7ff6a400b390 comp rx=0 tx=0).stop 2024-04-25T12:05:00.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.001+0000 7ff6af708640 1 -- 172.21.15.19:0/1113852295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a8103300 msgr2=0x7ff6a819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.001+0000 7ff6af708640 1 --2- 172.21.15.19:0/1113852295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a8103300 0x7ff6a819aac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff698009a50 tx=0x7ff6980042f0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.001+0000 7ff6af708640 1 -- 172.21.15.19:0/1113852295 shutdown_connections 2024-04-25T12:05:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.001+0000 7ff6af708640 1 --2- 172.21.15.19:0/1113852295 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff6940a3bb0 0x7ff6940a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.001+0000 7ff6af708640 1 --2- 172.21.15.19:0/1113852295 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6a8103cc0 0x7ff6a81a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.001+0000 7ff6af708640 1 --2- 172.21.15.19:0/1113852295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6a8103300 0x7ff6a819aac0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.001+0000 7ff6af708640 1 --2- 172.21.15.19:0/1113852295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6a8100290 0x7ff6a819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.001+0000 7ff6af708640 1 -- 172.21.15.19:0/1113852295 >> 172.21.15.19:0/1113852295 conn(0x7ff6a80fbb80 msgr2=0x7ff6a81010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.001+0000 7ff6af708640 1 -- 172.21.15.19:0/1113852295 shutdown_connections 2024-04-25T12:05:00.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.001+0000 7ff6af708640 1 -- 172.21.15.19:0/1113852295 wait complete. 2024-04-25T12:05:01.009 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T12:05:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.115+0000 7fe80864e640 1 -- 172.21.15.19:0/4235645289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe800101fe0 msgr2=0x7fe800102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.115+0000 7fe80864e640 1 --2- 172.21.15.19:0/4235645289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe800101fe0 0x7fe800102460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009900 tx=0x7fe7f002a410 comp rx=0 tx=0).stop 2024-04-25T12:05:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.115+0000 7fe80864e640 1 -- 172.21.15.19:0/4235645289 shutdown_connections 2024-04-25T12:05:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.115+0000 7fe80864e640 1 --2- 172.21.15.19:0/4235645289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8001029a0 0x7fe8001092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.115+0000 7fe80864e640 1 --2- 172.21.15.19:0/4235645289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe800101fe0 0x7fe800102460 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.115+0000 7fe80864e640 1 --2- 172.21.15.19:0/4235645289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe800100e90 0x7fe8001012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.115+0000 7fe80864e640 1 -- 172.21.15.19:0/4235645289 >> 172.21.15.19:0/4235645289 conn(0x7fe8000fc400 msgr2=0x7fe8000fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.115+0000 7fe80864e640 1 -- 172.21.15.19:0/4235645289 shutdown_connections 2024-04-25T12:05:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.115+0000 7fe80864e640 1 -- 172.21.15.19:0/4235645289 wait complete. 2024-04-25T12:05:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.116+0000 7fe80864e640 1 Processor -- start 2024-04-25T12:05:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.116+0000 7fe80864e640 1 -- start start 2024-04-25T12:05:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.116+0000 7fe80864e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe800100e90 0x7fe800198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.116+0000 7fe80864e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe800101fe0 0x7fe8001988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.116+0000 7fe80864e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8001029a0 0x7fe80019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.116+0000 7fe80864e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe80010cb20 con 0x7fe800101fe0 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.116+0000 7fe80864e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe80010c9a0 con 0x7fe800100e90 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.116+0000 7fe80864e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe80010cca0 con 0x7fe8001029a0 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.116+0000 7fe805be9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe800101fe0 0x7fe8001988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.116+0000 7fe805be9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe800101fe0 0x7fe8001988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46290/0 (socket says 172.21.15.19:46290) 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.116+0000 7fe805be9640 1 -- 172.21.15.19:0/68512596 learned_addr learned my addr 172.21.15.19:0/68512596 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.116+0000 7fe8063ea640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe800100e90 0x7fe800198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.116+0000 7fe806beb640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8001029a0 0x7fe80019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.117+0000 7fe8063ea640 1 -- 172.21.15.19:0/68512596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8001029a0 msgr2=0x7fe80019f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.117+0000 7fe8063ea640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8001029a0 0x7fe80019f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.117+0000 7fe8063ea640 1 -- 172.21.15.19:0/68512596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe800101fe0 msgr2=0x7fe8001988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.117+0000 7fe8063ea640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe800101fe0 0x7fe8001988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.117+0000 7fe8063ea640 1 -- 172.21.15.19:0/68512596 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80019ff20 con 0x7fe800100e90 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.117+0000 7fe806beb640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8001029a0 0x7fe80019f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:05:01.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.117+0000 7fe805be9640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe800101fe0 0x7fe8001988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:05:01.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.117+0000 7fe8063ea640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe800100e90 0x7fe800198370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00e9f0 tx=0x7fe7fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:01.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.117+0000 7fe7f77fe640 1 -- 172.21.15.19:0/68512596 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc00ccc0 con 0x7fe800100e90 2024-04-25T12:05:01.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.117+0000 7fe80864e640 1 -- 172.21.15.19:0/68512596 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8001a0210 con 0x7fe800100e90 2024-04-25T12:05:01.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.117+0000 7fe7f77fe640 1 -- 172.21.15.19:0/68512596 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe7fc010900 con 0x7fe800100e90 2024-04-25T12:05:01.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.117+0000 7fe7f77fe640 1 -- 172.21.15.19:0/68512596 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc018920 con 0x7fe800100e90 2024-04-25T12:05:01.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.117+0000 7fe80864e640 1 -- 172.21.15.19:0/68512596 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8001a0760 con 0x7fe800100e90 2024-04-25T12:05:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.119+0000 7fe80864e640 1 -- 172.21.15.19:0/68512596 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8001a09c0 con 0x7fe800100e90 2024-04-25T12:05:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.119+0000 7fe7f77fe640 1 -- 172.21.15.19:0/68512596 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe7fc010ab0 con 0x7fe800100e90 2024-04-25T12:05:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.120+0000 7fe7f77fe640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe7ec0a3bb0 0x7fe7ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.120+0000 7fe805be9640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe7ec0a3bb0 0x7fe7ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.120+0000 7fe7f77fe640 1 -- 172.21.15.19:0/68512596 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe7fc0d0220 con 0x7fe800100e90 2024-04-25T12:05:01.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.120+0000 7fe805be9640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe7ec0a3bb0 0x7fe7ec0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe800199810 tx=0x7fe7f0031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:01.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.125+0000 7fe7f77fe640 1 -- 172.21.15.19:0/68512596 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7fc09ca50 con 0x7fe800100e90 2024-04-25T12:05:01.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:01 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/785146057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:05:01.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:05:01.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:05:01.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:01.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:01 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1327938119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:01.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:01 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:05:01.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:01 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1113852295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:05:01.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.328+0000 7fe80864e640 1 -- 172.21.15.19:0/68512596 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe8001a0b90 con 0x7fe800100e90 2024-04-25T12:05:01.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.328+0000 7fe7f77fe640 1 -- 172.21.15.19:0/68512596 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7fe7fc0a0d50 con 0x7fe800100e90 2024-04-25T12:05:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:05:01.326 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-04-25T12:05:01.326 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:05:01.326 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-04-25T12:05:01.326 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-04-25T12:05:01.326 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:05:01.326 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-04-25T12:05:01.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.330+0000 7fe80864e640 1 -- 172.21.15.19:0/68512596 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe7ec0a3bb0 msgr2=0x7fe7ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:01.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.330+0000 7fe80864e640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe7ec0a3bb0 0x7fe7ec0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe800199810 tx=0x7fe7f0031510 comp rx=0 tx=0).stop 2024-04-25T12:05:01.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.330+0000 7fe80864e640 1 -- 172.21.15.19:0/68512596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe800100e90 msgr2=0x7fe800198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:01.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.330+0000 7fe80864e640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe800100e90 0x7fe800198370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00e9f0 tx=0x7fe7fc00eeb0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.330+0000 7fe80864e640 1 -- 172.21.15.19:0/68512596 shutdown_connections 2024-04-25T12:05:01.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.330+0000 7fe80864e640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe7ec0a3bb0 0x7fe7ec0a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.330+0000 7fe80864e640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8001029a0 0x7fe80019f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.330+0000 7fe80864e640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe800101fe0 0x7fe8001988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.330+0000 7fe80864e640 1 --2- 172.21.15.19:0/68512596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe800100e90 0x7fe800198370 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.330+0000 7fe80864e640 1 -- 172.21.15.19:0/68512596 >> 172.21.15.19:0/68512596 conn(0x7fe8000fc400 msgr2=0x7fe8000fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:01.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.330+0000 7fe80864e640 1 -- 172.21.15.19:0/68512596 shutdown_connections 2024-04-25T12:05:01.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.330+0000 7fe80864e640 1 -- 172.21.15.19:0/68512596 wait complete. 2024-04-25T12:05:01.339 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T12:05:01.339 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T12:05:01.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.456+0000 7fb8797fe640 1 -- 172.21.15.19:0/1377278602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874102a70 msgr2=0x7fb874104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:01.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.456+0000 7fb8797fe640 1 --2- 172.21.15.19:0/1377278602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874102a70 0x7fb874104e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb864009900 tx=0x7fb86402a410 comp rx=0 tx=0).stop 2024-04-25T12:05:01.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.456+0000 7fb8797fe640 1 -- 172.21.15.19:0/1377278602 shutdown_connections 2024-04-25T12:05:01.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.456+0000 7fb8797fe640 1 --2- 172.21.15.19:0/1377278602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb874107cd0 0x7fb87410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.456+0000 7fb8797fe640 1 --2- 172.21.15.19:0/1377278602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8741053a0 0x7fb874107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.456+0000 7fb8797fe640 1 --2- 172.21.15.19:0/1377278602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874102a70 0x7fb874104e60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.456+0000 7fb8797fe640 1 -- 172.21.15.19:0/1377278602 >> 172.21.15.19:0/1377278602 conn(0x7fb8740fc400 msgr2=0x7fb8740fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:01.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.456+0000 7fb8797fe640 1 -- 172.21.15.19:0/1377278602 shutdown_connections 2024-04-25T12:05:01.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.456+0000 7fb8797fe640 1 -- 172.21.15.19:0/1377278602 wait complete. 2024-04-25T12:05:01.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.457+0000 7fb8797fe640 1 Processor -- start 2024-04-25T12:05:01.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.457+0000 7fb8797fe640 1 -- start start 2024-04-25T12:05:01.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.457+0000 7fb8797fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb874102a70 0x7fb87419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:01.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.457+0000 7fb8797fe640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8741053a0 0x7fb87419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:01.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.457+0000 7fb8797fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874107cd0 0x7fb8741a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:01.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.457+0000 7fb8797fe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb87410dc10 con 0x7fb874107cd0 2024-04-25T12:05:01.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.457+0000 7fb8797fe640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb87410da90 con 0x7fb874102a70 2024-04-25T12:05:01.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.457+0000 7fb8797fe640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb87410dd90 con 0x7fb8741053a0 2024-04-25T12:05:01.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.457+0000 7fb8737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874107cd0 0x7fb8741a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:01.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb8737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874107cd0 0x7fb8741a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46308/0 (socket says 172.21.15.19:46308) 2024-04-25T12:05:01.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb8737fe640 1 -- 172.21.15.19:0/559380221 learned_addr learned my addr 172.21.15.19:0/559380221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:01.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb872ffd640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb874102a70 0x7fb87419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:01.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb8727fc640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8741053a0 0x7fb87419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:01.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb8737fe640 1 -- 172.21.15.19:0/559380221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8741053a0 msgr2=0x7fb87419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:01.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb8737fe640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8741053a0 0x7fb87419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb8737fe640 1 -- 172.21.15.19:0/559380221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb874102a70 msgr2=0x7fb87419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:01.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb8737fe640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb874102a70 0x7fb87419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:01.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb8737fe640 1 -- 172.21.15.19:0/559380221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8741a2140 con 0x7fb874107cd0 2024-04-25T12:05:01.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb8727fc640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8741053a0 0x7fb87419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:05:01.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb8737fe640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874107cd0 0x7fb8741a1ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00eaa0 tx=0x7fb86c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:01.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb85bfff640 1 -- 172.21.15.19:0/559380221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00cdb0 con 0x7fb874107cd0 2024-04-25T12:05:01.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb8797fe640 1 -- 172.21.15.19:0/559380221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8741a2430 con 0x7fb874107cd0 2024-04-25T12:05:01.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb85bfff640 1 -- 172.21.15.19:0/559380221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb86c005020 con 0x7fb874107cd0 2024-04-25T12:05:01.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb85bfff640 1 -- 172.21.15.19:0/559380221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c0116b0 con 0x7fb874107cd0 2024-04-25T12:05:01.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.458+0000 7fb8797fe640 1 -- 172.21.15.19:0/559380221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8741a2980 con 0x7fb874107cd0 2024-04-25T12:05:01.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.459+0000 7fb8797fe640 1 -- 172.21.15.19:0/559380221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8741097c0 con 0x7fb874107cd0 2024-04-25T12:05:01.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.461+0000 7fb85bfff640 1 -- 172.21.15.19:0/559380221 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb86c0051d0 con 0x7fb874107cd0 2024-04-25T12:05:01.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.461+0000 7fb85bfff640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb85c0a3bb0 0x7fb85c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:01.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.462+0000 7fb872ffd640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb85c0a3bb0 0x7fb85c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:01.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.462+0000 7fb85bfff640 1 -- 172.21.15.19:0/559380221 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb86c0d0150 con 0x7fb874107cd0 2024-04-25T12:05:01.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.462+0000 7fb872ffd640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb85c0a3bb0 0x7fb85c0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb864000c00 tx=0x7fb864031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:01.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.467+0000 7fb85bfff640 1 -- 172.21.15.19:0/559380221 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb86c016030 con 0x7fb874107cd0 2024-04-25T12:05:01.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:01 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/785146057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:05:01.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:05:01.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:05:01.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:01.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:01 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1327938119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:01.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:01 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:05:01.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:01 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1113852295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:05:01.647 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:01 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/785146057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:05:01.647 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:05:01.647 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:05:01.647 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:01.647 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:01 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1327938119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:01.648 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:01 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:05:01.648 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:01 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1113852295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:05:01.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:01.654+0000 7fb8797fe640 1 -- 172.21.15.19:0/559380221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb874062b50 con 0x7fb874107cd0 2024-04-25T12:05:02.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:02 smithi019 ceph-mon[29213]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:05:02.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:02 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/68512596' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T12:05:02.547 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:02 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/559380221' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T12:05:02.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.637+0000 7fb85bfff640 1 -- 172.21.15.19:0/559380221 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7fb86c09cfd0 con 0x7fb874107cd0 2024-04-25T12:05:02.635 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-04-25T12:05:02.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.639+0000 7fb8797fe640 1 -- 172.21.15.19:0/559380221 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb85c0a3bb0 msgr2=0x7fb85c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:02.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.639+0000 7fb8797fe640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb85c0a3bb0 0x7fb85c0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb864000c00 tx=0x7fb864031510 comp rx=0 tx=0).stop 2024-04-25T12:05:02.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.639+0000 7fb8797fe640 1 -- 172.21.15.19:0/559380221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874107cd0 msgr2=0x7fb8741a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:02.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.639+0000 7fb8797fe640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874107cd0 0x7fb8741a1ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00eaa0 tx=0x7fb86c00ef60 comp rx=0 tx=0).stop 2024-04-25T12:05:02.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.639+0000 7fb8797fe640 1 -- 172.21.15.19:0/559380221 shutdown_connections 2024-04-25T12:05:02.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.639+0000 7fb8797fe640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb85c0a3bb0 0x7fb85c0a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:02.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.639+0000 7fb8797fe640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb874107cd0 0x7fb8741a1ad0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:02.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.639+0000 7fb8797fe640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8741053a0 0x7fb87419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:02.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.639+0000 7fb8797fe640 1 --2- 172.21.15.19:0/559380221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb874102a70 0x7fb87419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:02.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.639+0000 7fb8797fe640 1 -- 172.21.15.19:0/559380221 >> 172.21.15.19:0/559380221 conn(0x7fb8740fc400 msgr2=0x7fb8741021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:02.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.639+0000 7fb8797fe640 1 -- 172.21.15.19:0/559380221 shutdown_connections 2024-04-25T12:05:02.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.639+0000 7fb8797fe640 1 -- 172.21.15.19:0/559380221 wait complete. 2024-04-25T12:05:02.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T12:05:02.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:02 smithi169 ceph-mon[32710]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:05:02.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:02 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/68512596' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T12:05:02.655 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:02 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/559380221' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T12:05:02.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:02 smithi150 ceph-mon[32250]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:05:02.662 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:02 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/68512596' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T12:05:02.663 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:02 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/559380221' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T12:05:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.765+0000 7fca662cb640 1 -- 172.21.15.19:0/1139560474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca601029a0 msgr2=0x7fca601092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.765+0000 7fca662cb640 1 --2- 172.21.15.19:0/1139560474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca601029a0 0x7fca601092c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fca58009ca0 tx=0x7fca5802a8c0 comp rx=0 tx=0).stop 2024-04-25T12:05:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.766+0000 7fca662cb640 1 -- 172.21.15.19:0/1139560474 shutdown_connections 2024-04-25T12:05:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.766+0000 7fca662cb640 1 --2- 172.21.15.19:0/1139560474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca601029a0 0x7fca601092c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.766+0000 7fca662cb640 1 --2- 172.21.15.19:0/1139560474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca60101fe0 0x7fca60102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.766+0000 7fca662cb640 1 --2- 172.21.15.19:0/1139560474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca60100e90 0x7fca601012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.766+0000 7fca662cb640 1 -- 172.21.15.19:0/1139560474 >> 172.21.15.19:0/1139560474 conn(0x7fca600fc400 msgr2=0x7fca600fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.766+0000 7fca662cb640 1 -- 172.21.15.19:0/1139560474 shutdown_connections 2024-04-25T12:05:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.766+0000 7fca662cb640 1 -- 172.21.15.19:0/1139560474 wait complete. 2024-04-25T12:05:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.766+0000 7fca662cb640 1 Processor -- start 2024-04-25T12:05:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.766+0000 7fca662cb640 1 -- start start 2024-04-25T12:05:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.766+0000 7fca662cb640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca60100e90 0x7fca60104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.766+0000 7fca662cb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca60101fe0 0x7fca60103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.766+0000 7fca662cb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca601029a0 0x7fca60103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.766+0000 7fca662cb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca6010cb30 con 0x7fca601029a0 2024-04-25T12:05:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca662cb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fca6010c9b0 con 0x7fca60101fe0 2024-04-25T12:05:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca662cb640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fca6010ccb0 con 0x7fca60100e90 2024-04-25T12:05:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca64868640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca601029a0 0x7fca60103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca64868640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca601029a0 0x7fca60103990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38854/0 (socket says 172.21.15.19:38854) 2024-04-25T12:05:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca64868640 1 -- 172.21.15.19:0/1682370808 learned_addr learned my addr 172.21.15.19:0/1682370808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca5f7fe640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca60101fe0 0x7fca60103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca5ffff640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca60100e90 0x7fca60104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca64868640 1 -- 172.21.15.19:0/1682370808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca60100e90 msgr2=0x7fca60104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca64868640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca60100e90 0x7fca60104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca64868640 1 -- 172.21.15.19:0/1682370808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca60101fe0 msgr2=0x7fca60103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca64868640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca60101fe0 0x7fca60103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca64868640 1 -- 172.21.15.19:0/1682370808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca60104220 con 0x7fca601029a0 2024-04-25T12:05:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca5ffff640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca60100e90 0x7fca60104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:05:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca64868640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca601029a0 0x7fca60103990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fca58009dc0 tx=0x7fca58031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca5d7fa640 1 -- 172.21.15.19:0/1682370808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58003bf0 con 0x7fca601029a0 2024-04-25T12:05:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca5d7fa640 1 -- 172.21.15.19:0/1682370808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fca58003d90 con 0x7fca601029a0 2024-04-25T12:05:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca5d7fa640 1 -- 172.21.15.19:0/1682370808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5802fcd0 con 0x7fca601029a0 2024-04-25T12:05:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca662cb640 1 -- 172.21.15.19:0/1682370808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca601a2360 con 0x7fca601029a0 2024-04-25T12:05:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.767+0000 7fca662cb640 1 -- 172.21.15.19:0/1682370808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca601a2900 con 0x7fca601029a0 2024-04-25T12:05:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.768+0000 7fca662cb640 1 -- 172.21.15.19:0/1682370808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca601a2bc0 con 0x7fca601029a0 2024-04-25T12:05:02.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.770+0000 7fca5d7fa640 1 -- 172.21.15.19:0/1682370808 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fca58004040 con 0x7fca601029a0 2024-04-25T12:05:02.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.770+0000 7fca5d7fa640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fca480a3c30 0x7fca480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:02.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.770+0000 7fca5ffff640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fca480a3c30 0x7fca480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:02.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.770+0000 7fca5d7fa640 1 -- 172.21.15.19:0/1682370808 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7fca580f18d0 con 0x7fca601029a0 2024-04-25T12:05:02.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.771+0000 7fca5ffff640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fca480a3c30 0x7fca480a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fca50005fd0 tx=0x7fca50005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:02.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.774+0000 7fca5d7fa640 1 -- 172.21.15.19:0/1682370808 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca580be5e0 con 0x7fca601029a0 2024-04-25T12:05:02.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:02.965+0000 7fca662cb640 1 -- 172.21.15.19:0/1682370808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fca601a31d0 con 0x7fca601029a0 2024-04-25T12:05:03.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.639+0000 7fca5d7fa640 1 -- 172.21.15.19:0/1682370808 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7fca580bdee0 con 0x7fca601029a0 2024-04-25T12:05:03.637 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-04-25T12:05:03.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.642+0000 7fca662cb640 1 -- 172.21.15.19:0/1682370808 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fca480a3c30 msgr2=0x7fca480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:03.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.642+0000 7fca662cb640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fca480a3c30 0x7fca480a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fca50005fd0 tx=0x7fca50005ea0 comp rx=0 tx=0).stop 2024-04-25T12:05:03.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.642+0000 7fca662cb640 1 -- 172.21.15.19:0/1682370808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca601029a0 msgr2=0x7fca60103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:03.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.642+0000 7fca662cb640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca601029a0 0x7fca60103990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fca58009dc0 tx=0x7fca58031840 comp rx=0 tx=0).stop 2024-04-25T12:05:03.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.642+0000 7fca662cb640 1 -- 172.21.15.19:0/1682370808 shutdown_connections 2024-04-25T12:05:03.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.642+0000 7fca662cb640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fca480a3c30 0x7fca480a60f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:03.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.642+0000 7fca662cb640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca601029a0 0x7fca60103990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:03.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.642+0000 7fca662cb640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca60101fe0 0x7fca60103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:03.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.642+0000 7fca662cb640 1 --2- 172.21.15.19:0/1682370808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca60100e90 0x7fca60104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:03.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.642+0000 7fca662cb640 1 -- 172.21.15.19:0/1682370808 >> 172.21.15.19:0/1682370808 conn(0x7fca600fc400 msgr2=0x7fca600fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:03.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.642+0000 7fca662cb640 1 -- 172.21.15.19:0/1682370808 shutdown_connections 2024-04-25T12:05:03.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.642+0000 7fca662cb640 1 -- 172.21.15.19:0/1682370808 wait complete. 2024-04-25T12:05:03.660 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T12:05:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.773+0000 7f8a696e8640 1 -- 172.21.15.19:0/2917926782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a64103b60 msgr2=0x7f8a6410a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.773+0000 7f8a696e8640 1 --2- 172.21.15.19:0/2917926782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a64103b60 0x7f8a6410a3f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009900 tx=0x7f8a5c02a410 comp rx=0 tx=0).stop 2024-04-25T12:05:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.773+0000 7f8a696e8640 1 -- 172.21.15.19:0/2917926782 shutdown_connections 2024-04-25T12:05:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.773+0000 7f8a696e8640 1 --2- 172.21.15.19:0/2917926782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a64103b60 0x7f8a6410a3f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.773+0000 7f8a696e8640 1 --2- 172.21.15.19:0/2917926782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a641031a0 0x7f8a64103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.773+0000 7f8a696e8640 1 --2- 172.21.15.19:0/2917926782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a64101fb0 0x7f8a641023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.773+0000 7f8a696e8640 1 -- 172.21.15.19:0/2917926782 >> 172.21.15.19:0/2917926782 conn(0x7f8a640fd520 msgr2=0x7f8a640ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.773+0000 7f8a696e8640 1 -- 172.21.15.19:0/2917926782 shutdown_connections 2024-04-25T12:05:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.773+0000 7f8a696e8640 1 -- 172.21.15.19:0/2917926782 wait complete. 2024-04-25T12:05:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.773+0000 7f8a696e8640 1 Processor -- start 2024-04-25T12:05:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.773+0000 7f8a696e8640 1 -- start start 2024-04-25T12:05:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a696e8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a64101fb0 0x7f8a64198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a696e8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a641031a0 0x7f8a64198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a696e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a64103b60 0x7f8a6419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a696e8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6410dc90 con 0x7f8a64103b60 2024-04-25T12:05:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a696e8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6410db10 con 0x7f8a641031a0 2024-04-25T12:05:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a696e8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6410de10 con 0x7f8a64101fb0 2024-04-25T12:05:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a637fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a64103b60 0x7f8a6419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a637fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a64103b60 0x7f8a6419f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38878/0 (socket says 172.21.15.19:38878) 2024-04-25T12:05:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a637fe640 1 -- 172.21.15.19:0/354208802 learned_addr learned my addr 172.21.15.19:0/354208802 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a62ffd640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a64101fb0 0x7f8a64198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a627fc640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a641031a0 0x7f8a64198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a637fe640 1 -- 172.21.15.19:0/354208802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a64101fb0 msgr2=0x7f8a64198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a637fe640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a64101fb0 0x7f8a64198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a637fe640 1 -- 172.21.15.19:0/354208802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a641031a0 msgr2=0x7f8a64198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a637fe640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a641031a0 0x7f8a64198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a637fe640 1 -- 172.21.15.19:0/354208802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6419ff30 con 0x7f8a64103b60 2024-04-25T12:05:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a62ffd640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a64101fb0 0x7f8a64198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:05:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a627fc640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a641031a0 0x7f8a64198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:05:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a637fe640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a64103b60 0x7f8a6419f8c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c005940 tx=0x7f8a5c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a5bfff640 1 -- 172.21.15.19:0/354208802 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c03f030 con 0x7f8a64103b60 2024-04-25T12:05:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a696e8640 1 -- 172.21.15.19:0/354208802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a641a01c0 con 0x7f8a64103b60 2024-04-25T12:05:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a5bfff640 1 -- 172.21.15.19:0/354208802 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8a5c003cd0 con 0x7f8a64103b60 2024-04-25T12:05:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a5bfff640 1 -- 172.21.15.19:0/354208802 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c035070 con 0x7f8a64103b60 2024-04-25T12:05:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.774+0000 7f8a696e8640 1 -- 172.21.15.19:0/354208802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a641a0780 con 0x7f8a64103b60 2024-04-25T12:05:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.775+0000 7f8a696e8640 1 -- 172.21.15.19:0/354208802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a641092f0 con 0x7f8a64103b60 2024-04-25T12:05:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.777+0000 7f8a5bfff640 1 -- 172.21.15.19:0/354208802 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8a5c042030 con 0x7f8a64103b60 2024-04-25T12:05:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.777+0000 7f8a5bfff640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8a500a3bb0 0x7f8a500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.777+0000 7f8a62ffd640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8a500a3bb0 0x7f8a500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.777+0000 7f8a5bfff640 1 -- 172.21.15.19:0/354208802 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f8a5c0f0c00 con 0x7f8a64103b60 2024-04-25T12:05:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.778+0000 7f8a62ffd640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8a500a3bb0 0x7f8a500a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a64103000 tx=0x7f8a54009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:03.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:03.781+0000 7f8a5bfff640 1 -- 172.21.15.19:0/354208802 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a5c0bd790 con 0x7f8a64103b60 2024-04-25T12:05:03.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:03 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:03 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:03 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:03 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:03 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/559380221' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T12:05:03.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:03 smithi019 ceph-mon[29213]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T12:05:03.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:03 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:03 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:03 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:05:03.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:03 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:05:03.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:03 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:03 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1682370808' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T12:05:03.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:03 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:03 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:03 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:03 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:03 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/559380221' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T12:05:03.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:03 smithi169 ceph-mon[32710]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T12:05:03.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:03 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:03 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:03 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:05:03.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:03 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:05:03.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:03 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:03 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1682370808' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T12:05:03.912 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:03 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:03 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:03 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:03 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:03 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/559380221' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T12:05:03.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:03 smithi150 ceph-mon[32250]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T12:05:03.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:03 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:03 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:03 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:05:03.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:03 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:05:03.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:03 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:05:03.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:03 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1682370808' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T12:05:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.007+0000 7f8a696e8640 1 -- 172.21.15.19:0/354208802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8a64062b50 con 0x7f8a64103b60 2024-04-25T12:05:04.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.674+0000 7f8a5bfff640 1 -- 172.21.15.19:0/354208802 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f8a5c0bd090 con 0x7f8a64103b60 2024-04-25T12:05:04.672 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T12:05:04.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.679+0000 7f8a696e8640 1 -- 172.21.15.19:0/354208802 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8a500a3bb0 msgr2=0x7f8a500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:04.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.679+0000 7f8a696e8640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8a500a3bb0 0x7f8a500a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a64103000 tx=0x7f8a54009570 comp rx=0 tx=0).stop 2024-04-25T12:05:04.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.679+0000 7f8a696e8640 1 -- 172.21.15.19:0/354208802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a64103b60 msgr2=0x7f8a6419f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:04.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.679+0000 7f8a696e8640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a64103b60 0x7f8a6419f8c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c005940 tx=0x7f8a5c0043b0 comp rx=0 tx=0).stop 2024-04-25T12:05:04.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.679+0000 7f8a696e8640 1 -- 172.21.15.19:0/354208802 shutdown_connections 2024-04-25T12:05:04.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.679+0000 7f8a696e8640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8a500a3bb0 0x7f8a500a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:04.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.679+0000 7f8a696e8640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a64103b60 0x7f8a6419f8c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:04.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.679+0000 7f8a696e8640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a641031a0 0x7f8a64198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:04.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.679+0000 7f8a696e8640 1 --2- 172.21.15.19:0/354208802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a64101fb0 0x7f8a64198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:04.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.679+0000 7f8a696e8640 1 -- 172.21.15.19:0/354208802 >> 172.21.15.19:0/354208802 conn(0x7f8a640fd520 msgr2=0x7f8a641064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:04.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.679+0000 7f8a696e8640 1 -- 172.21.15.19:0/354208802 shutdown_connections 2024-04-25T12:05:04.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.679+0000 7f8a696e8640 1 -- 172.21.15.19:0/354208802 wait complete. 2024-04-25T12:05:04.692 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:05:04.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.803+0000 7f6d938c2640 1 -- 172.21.15.19:0/2880381142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d8c100e60 msgr2=0x7f6d8c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:04.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.803+0000 7f6d938c2640 1 --2- 172.21.15.19:0/2880381142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d8c100e60 0x7f6d8c101280 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009900 tx=0x7f6d8802a410 comp rx=0 tx=0).stop 2024-04-25T12:05:04.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.803+0000 7f6d938c2640 1 -- 172.21.15.19:0/2880381142 shutdown_connections 2024-04-25T12:05:04.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.803+0000 7f6d938c2640 1 --2- 172.21.15.19:0/2880381142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d8c102a00 0x7f6d8c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:04.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.803+0000 7f6d938c2640 1 --2- 172.21.15.19:0/2880381142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d8c102040 0x7f6d8c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:04.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.803+0000 7f6d938c2640 1 --2- 172.21.15.19:0/2880381142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d8c100e60 0x7f6d8c101280 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:04.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.803+0000 7f6d938c2640 1 -- 172.21.15.19:0/2880381142 >> 172.21.15.19:0/2880381142 conn(0x7f6d8c0fc440 msgr2=0x7f6d8c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:04.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.803+0000 7f6d938c2640 1 -- 172.21.15.19:0/2880381142 shutdown_connections 2024-04-25T12:05:04.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.803+0000 7f6d938c2640 1 -- 172.21.15.19:0/2880381142 wait complete. 2024-04-25T12:05:04.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.803+0000 7f6d938c2640 1 Processor -- start 2024-04-25T12:05:04.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d938c2640 1 -- start start 2024-04-25T12:05:04.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d938c2640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d8c100e60 0x7f6d8c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:04.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d938c2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d8c102040 0x7f6d8c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d938c2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d8c102a00 0x7f6d8c19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d938c2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c10dc30 con 0x7f6d8c102040 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d938c2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c10dab0 con 0x7f6d8c102a00 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d938c2640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c10ddb0 con 0x7f6d8c100e60 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d90e5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d8c102040 0x7f6d8c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d90e5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d8c102040 0x7f6d8c198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38908/0 (socket says 172.21.15.19:38908) 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d90e5d640 1 -- 172.21.15.19:0/2817017125 learned_addr learned my addr 172.21.15.19:0/2817017125 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d91e5f640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d8c102a00 0x7f6d8c19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d9165e640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d8c100e60 0x7f6d8c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d90e5d640 1 -- 172.21.15.19:0/2817017125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d8c100e60 msgr2=0x7f6d8c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d90e5d640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d8c100e60 0x7f6d8c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d90e5d640 1 -- 172.21.15.19:0/2817017125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d8c102a00 msgr2=0x7f6d8c19f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d90e5d640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d8c102a00 0x7f6d8c19f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d90e5d640 1 -- 172.21.15.19:0/2817017125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c19ff20 con 0x7f6d8c102040 2024-04-25T12:05:04.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d9165e640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d8c100e60 0x7f6d8c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:05:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.804+0000 7f6d90e5d640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d8c102040 0x7f6d8c198840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00d950 tx=0x7f6d7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.805+0000 7f6d827fc640 1 -- 172.21.15.19:0/2817017125 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c017070 con 0x7f6d8c102040 2024-04-25T12:05:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.805+0000 7f6d827fc640 1 -- 172.21.15.19:0/2817017125 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6d7c009c30 con 0x7f6d8c102040 2024-04-25T12:05:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.805+0000 7f6d938c2640 1 -- 172.21.15.19:0/2817017125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8c1a0210 con 0x7f6d8c102040 2024-04-25T12:05:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.805+0000 7f6d827fc640 1 -- 172.21.15.19:0/2817017125 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00f6b0 con 0x7f6d8c102040 2024-04-25T12:05:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.805+0000 7f6d938c2640 1 -- 172.21.15.19:0/2817017125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8c1a0760 con 0x7f6d8c102040 2024-04-25T12:05:04.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.806+0000 7f6d938c2640 1 -- 172.21.15.19:0/2817017125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8c1082a0 con 0x7f6d8c102040 2024-04-25T12:05:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.807+0000 7f6d827fc640 1 -- 172.21.15.19:0/2817017125 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6d7c0109c0 con 0x7f6d8c102040 2024-04-25T12:05:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.808+0000 7f6d827fc640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6d780a3bb0 0x7f6d780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.808+0000 7f6d9165e640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6d780a3bb0 0x7f6d780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.808+0000 7f6d827fc640 1 -- 172.21.15.19:0/2817017125 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f6d7c0cff20 con 0x7f6d8c102040 2024-04-25T12:05:04.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.808+0000 7f6d9165e640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6d780a3bb0 0x7f6d780a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6d88000c00 tx=0x7f6d88031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:04.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.811+0000 7f6d827fc640 1 -- 172.21.15.19:0/2817017125 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d7c009de0 con 0x7f6d8c102040 2024-04-25T12:05:04.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:04 smithi169 ceph-mon[32710]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:05:04.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:04 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1682370808' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T12:05:04.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:04 smithi169 ceph-mon[32710]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T12:05:04.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:04 smithi169 ceph-mon[32710]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T12:05:04.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:04 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/354208802' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T12:05:04.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:04 smithi169 ceph-mon[32710]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-25T12:05:04.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:04 smithi150 ceph-mon[32250]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:05:04.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:04 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1682370808' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T12:05:04.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:04 smithi150 ceph-mon[32250]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T12:05:04.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:04 smithi150 ceph-mon[32250]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T12:05:04.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:04 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/354208802' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T12:05:04.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:04 smithi150 ceph-mon[32250]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-25T12:05:04.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:04.999+0000 7f6d938c2640 1 -- 172.21.15.19:0/2817017125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d8c02d790 con 0x7f6d8c102040 2024-04-25T12:05:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.001+0000 7f6d827fc640 1 -- 172.21.15.19:0/2817017125 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22216 (secure 0 0 0) 0x7f6d7c09ca50 con 0x7f6d8c102040 2024-04-25T12:05:04.999 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:05:04.999 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":84,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:05:04.008718+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:05:01.658354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:05:02.969078+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:05:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.003+0000 7f6d938c2640 1 -- 172.21.15.19:0/2817017125 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6d780a3bb0 msgr2=0x7f6d780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.003+0000 7f6d938c2640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6d780a3bb0 0x7f6d780a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6d88000c00 tx=0x7f6d88031510 comp rx=0 tx=0).stop 2024-04-25T12:05:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.003+0000 7f6d938c2640 1 -- 172.21.15.19:0/2817017125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d8c102040 msgr2=0x7f6d8c198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.003+0000 7f6d938c2640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d8c102040 0x7f6d8c198840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00d950 tx=0x7f6d7c00de10 comp rx=0 tx=0).stop 2024-04-25T12:05:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.003+0000 7f6d938c2640 1 -- 172.21.15.19:0/2817017125 shutdown_connections 2024-04-25T12:05:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.003+0000 7f6d938c2640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6d780a3bb0 0x7f6d780a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.003+0000 7f6d938c2640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d8c102a00 0x7f6d8c19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.003+0000 7f6d938c2640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d8c102040 0x7f6d8c198840 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.003+0000 7f6d938c2640 1 --2- 172.21.15.19:0/2817017125 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d8c100e60 0x7f6d8c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.003+0000 7f6d938c2640 1 -- 172.21.15.19:0/2817017125 >> 172.21.15.19:0/2817017125 conn(0x7f6d8c0fc440 msgr2=0x7f6d8c105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.003+0000 7f6d938c2640 1 -- 172.21.15.19:0/2817017125 shutdown_connections 2024-04-25T12:05:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.003+0000 7f6d938c2640 1 -- 172.21.15.19:0/2817017125 wait complete. 2024-04-25T12:05:05.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:04 smithi019 ceph-mon[29213]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:05:05.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:04 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1682370808' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T12:05:05.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:04 smithi019 ceph-mon[29213]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T12:05:05.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:04 smithi019 ceph-mon[29213]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T12:05:05.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:04 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/354208802' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T12:05:05.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:04 smithi019 ceph-mon[29213]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-25T12:05:05.012 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:05:05.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.117+0000 7f38d595d640 1 -- 172.21.15.19:0/1455765193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38d0105360 msgr2=0x7f38d0107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:05.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.117+0000 7f38d595d640 1 --2- 172.21.15.19:0/1455765193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38d0105360 0x7f38d0107750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009900 tx=0x7f38c402a410 comp rx=0 tx=0).stop 2024-04-25T12:05:05.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.117+0000 7f38d595d640 1 -- 172.21.15.19:0/1455765193 shutdown_connections 2024-04-25T12:05:05.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.117+0000 7f38d595d640 1 --2- 172.21.15.19:0/1455765193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d0107c90 0x7f38d010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.117+0000 7f38d595d640 1 --2- 172.21.15.19:0/1455765193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38d0105360 0x7f38d0107750 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.117+0000 7f38d595d640 1 --2- 172.21.15.19:0/1455765193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38d0102a30 0x7f38d0104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.117+0000 7f38d595d640 1 -- 172.21.15.19:0/1455765193 >> 172.21.15.19:0/1455765193 conn(0x7f38d00fc440 msgr2=0x7f38d00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:05.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.117+0000 7f38d595d640 1 -- 172.21.15.19:0/1455765193 shutdown_connections 2024-04-25T12:05:05.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.117+0000 7f38d595d640 1 -- 172.21.15.19:0/1455765193 wait complete. 2024-04-25T12:05:05.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38d595d640 1 Processor -- start 2024-04-25T12:05:05.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38d595d640 1 -- start start 2024-04-25T12:05:05.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38d595d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d0102a30 0x7f38d01023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:05.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38d595d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38d0105360 0x7f38d01009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:05.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38d595d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38d0107c90 0x7f38d0100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:05.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38d595d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f38d010dc10 con 0x7f38d0105360 2024-04-25T12:05:05.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38d595d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38d010da90 con 0x7f38d0107c90 2024-04-25T12:05:05.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38d595d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f38d010dd90 con 0x7f38d0102a30 2024-04-25T12:05:05.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38ce7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38d0105360 0x7f38d01009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:05.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38ce7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38d0105360 0x7f38d01009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38924/0 (socket says 172.21.15.19:38924) 2024-04-25T12:05:05.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38ce7fc640 1 -- 172.21.15.19:0/2498485123 learned_addr learned my addr 172.21.15.19:0/2498485123 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:05.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38ceffd640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d0102a30 0x7f38d01023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:05.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38cf7fe640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38d0107c90 0x7f38d0100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:05.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38ce7fc640 1 -- 172.21.15.19:0/2498485123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d0102a30 msgr2=0x7f38d01023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38ce7fc640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d0102a30 0x7f38d01023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38ce7fc640 1 -- 172.21.15.19:0/2498485123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38d0107c90 msgr2=0x7f38d0100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38ce7fc640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38d0107c90 0x7f38d0100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38ce7fc640 1 -- 172.21.15.19:0/2498485123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d01017b0 con 0x7f38d0105360 2024-04-25T12:05:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.118+0000 7f38cf7fe640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38d0107c90 0x7f38d0100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:05:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.119+0000 7f38ce7fc640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38d0105360 0x7f38d01009e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f38c4006010 tx=0x7f38c40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.119+0000 7f38d495b640 1 -- 172.21.15.19:0/2498485123 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c403f030 con 0x7f38d0105360 2024-04-25T12:05:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.119+0000 7f38d495b640 1 -- 172.21.15.19:0/2498485123 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f38c4003cd0 con 0x7f38d0105360 2024-04-25T12:05:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.119+0000 7f38d595d640 1 -- 172.21.15.19:0/2498485123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d01a23b0 con 0x7f38d0105360 2024-04-25T12:05:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.119+0000 7f38d495b640 1 -- 172.21.15.19:0/2498485123 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4035070 con 0x7f38d0105360 2024-04-25T12:05:05.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.119+0000 7f38d595d640 1 -- 172.21.15.19:0/2498485123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d01a2900 con 0x7f38d0105360 2024-04-25T12:05:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.120+0000 7f38d595d640 1 -- 172.21.15.19:0/2498485123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38d00fdf30 con 0x7f38d0105360 2024-04-25T12:05:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.121+0000 7f38d495b640 1 -- 172.21.15.19:0/2498485123 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f38c4042030 con 0x7f38d0105360 2024-04-25T12:05:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.121+0000 7f38d495b640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f38b80a3bb0 0x7f38b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.122+0000 7f38ceffd640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f38b80a3bb0 0x7f38b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:05.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.122+0000 7f38d495b640 1 -- 172.21.15.19:0/2498485123 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f38c40f0a40 con 0x7f38d0105360 2024-04-25T12:05:05.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.122+0000 7f38ceffd640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f38b80a3bb0 0x7f38b80a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f38c00058c0 tx=0x7f38c000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:05.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.126+0000 7f38d495b640 1 -- 172.21.15.19:0/2498485123 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38c40bd570 con 0x7f38d0105360 2024-04-25T12:05:05.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.327+0000 7f38d595d640 1 -- 172.21.15.19:0/2498485123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38d002d810 con 0x7f38d0105360 2024-04-25T12:05:05.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.328+0000 7f38d495b640 1 -- 172.21.15.19:0/2498485123 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22216 (secure 0 0 0) 0x7f38c40bce70 con 0x7f38d0105360 2024-04-25T12:05:05.326 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:05:05.326 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":84,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:05:04.008718+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:05:01.658354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:05:02.969078+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:05:05.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.331+0000 7f38d595d640 1 -- 172.21.15.19:0/2498485123 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f38b80a3bb0 msgr2=0x7f38b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:05.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.331+0000 7f38d595d640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f38b80a3bb0 0x7f38b80a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f38c00058c0 tx=0x7f38c000a2d0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.331+0000 7f38d595d640 1 -- 172.21.15.19:0/2498485123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38d0105360 msgr2=0x7f38d01009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:05.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.331+0000 7f38d595d640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38d0105360 0x7f38d01009e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f38c4006010 tx=0x7f38c40043b0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.331+0000 7f38d595d640 1 -- 172.21.15.19:0/2498485123 shutdown_connections 2024-04-25T12:05:05.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.331+0000 7f38d595d640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f38b80a3bb0 0x7f38b80a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.331+0000 7f38d595d640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38d0107c90 0x7f38d0100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.331+0000 7f38d595d640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38d0105360 0x7f38d01009e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.331+0000 7f38d595d640 1 --2- 172.21.15.19:0/2498485123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f38d0102a30 0x7f38d01023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.331+0000 7f38d595d640 1 -- 172.21.15.19:0/2498485123 >> 172.21.15.19:0/2498485123 conn(0x7f38d00fc440 msgr2=0x7f38d0103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:05.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.331+0000 7f38d595d640 1 -- 172.21.15.19:0/2498485123 shutdown_connections 2024-04-25T12:05:05.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.331+0000 7f38d595d640 1 -- 172.21.15.19:0/2498485123 wait complete. 2024-04-25T12:05:05.340 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T12:05:05.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.447+0000 7f4908bcc640 1 -- 172.21.15.19:0/820907436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4904100e90 msgr2=0x7f49041012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.447+0000 7f4908bcc640 1 --2- 172.21.15.19:0/820907436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4904100e90 0x7f49041012b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009900 tx=0x7f48f402a410 comp rx=0 tx=0).stop 2024-04-25T12:05:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.448+0000 7f4908bcc640 1 -- 172.21.15.19:0/820907436 shutdown_connections 2024-04-25T12:05:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.448+0000 7f4908bcc640 1 --2- 172.21.15.19:0/820907436 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49041029a0 0x7f49041092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.448+0000 7f4908bcc640 1 --2- 172.21.15.19:0/820907436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4904101fe0 0x7f4904102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.448+0000 7f4908bcc640 1 --2- 172.21.15.19:0/820907436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4904100e90 0x7f49041012b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.448+0000 7f4908bcc640 1 -- 172.21.15.19:0/820907436 >> 172.21.15.19:0/820907436 conn(0x7f49040fc400 msgr2=0x7f49040fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.448+0000 7f4908bcc640 1 -- 172.21.15.19:0/820907436 shutdown_connections 2024-04-25T12:05:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.448+0000 7f4908bcc640 1 -- 172.21.15.19:0/820907436 wait complete. 2024-04-25T12:05:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.448+0000 7f4908bcc640 1 Processor -- start 2024-04-25T12:05:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.448+0000 7f4908bcc640 1 -- start start 2024-04-25T12:05:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f4908bcc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4904100e90 0x7f4904198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f4908bcc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4904101fe0 0x7f4904198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:05.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f4908bcc640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49041029a0 0x7f490419f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f4908bcc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f490410dd70 con 0x7f4904100e90 2024-04-25T12:05:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f4908bcc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f490410dbf0 con 0x7f4904101fe0 2024-04-25T12:05:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f4908bcc640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f490410def0 con 0x7f49041029a0 2024-04-25T12:05:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f490259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4904100e90 0x7f4904198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f490259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4904100e90 0x7f4904198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38950/0 (socket says 172.21.15.19:38950) 2024-04-25T12:05:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f490259c640 1 -- 172.21.15.19:0/4065298233 learned_addr learned my addr 172.21.15.19:0/4065298233 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f4901d9b640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4904101fe0 0x7f4904198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f490259c640 1 -- 172.21.15.19:0/4065298233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49041029a0 msgr2=0x7f490419f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f4902d9d640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49041029a0 0x7f490419f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f490259c640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49041029a0 0x7f490419f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f490259c640 1 -- 172.21.15.19:0/4065298233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4904101fe0 msgr2=0x7f4904198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f490259c640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4904101fe0 0x7f4904198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:05.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f490259c640 1 -- 172.21.15.19:0/4065298233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49041a0060 con 0x7f4904100e90 2024-04-25T12:05:05.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f4901d9b640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4904101fe0 0x7f4904198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:05:05.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f490259c640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4904100e90 0x7f4904198440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f48f4031570 tx=0x7f48f4032d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:05.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f48f37fe640 1 -- 172.21.15.19:0/4065298233 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f403f030 con 0x7f4904100e90 2024-04-25T12:05:05.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f48f37fe640 1 -- 172.21.15.19:0/4065298233 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f48f4003d90 con 0x7f4904100e90 2024-04-25T12:05:05.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f4908bcc640 1 -- 172.21.15.19:0/4065298233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49041a02f0 con 0x7f4904100e90 2024-04-25T12:05:05.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f48f37fe640 1 -- 172.21.15.19:0/4065298233 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4035070 con 0x7f4904100e90 2024-04-25T12:05:05.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.449+0000 7f4908bcc640 1 -- 172.21.15.19:0/4065298233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49041a0860 con 0x7f4904100e90 2024-04-25T12:05:05.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.450+0000 7f4908bcc640 1 -- 172.21.15.19:0/4065298233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4904108270 con 0x7f4904100e90 2024-04-25T12:05:05.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.451+0000 7f48f37fe640 1 -- 172.21.15.19:0/4065298233 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f48f4042030 con 0x7f4904100e90 2024-04-25T12:05:05.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.452+0000 7f48f37fe640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f48ec0a3bb0 0x7f48ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:05.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.452+0000 7f4901d9b640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f48ec0a3bb0 0x7f48ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:05.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.452+0000 7f48f37fe640 1 -- 172.21.15.19:0/4065298233 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f48f40f0a20 con 0x7f4904100e90 2024-04-25T12:05:05.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.453+0000 7f4901d9b640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f48ec0a3bb0 0x7f48ec0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4904199950 tx=0x7f48f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:05.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.456+0000 7f48f37fe640 1 -- 172.21.15.19:0/4065298233 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48f40bd550 con 0x7f4904100e90 2024-04-25T12:05:05.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:05.686+0000 7f4908bcc640 1 -- 172.21.15.19:0/4065298233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f490402d5c0 con 0x7f4904100e90 2024-04-25T12:05:05.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:05 smithi169 ceph-mon[32710]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T12:05:05.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:05 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/354208802' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T12:05:05.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:05 smithi169 ceph-mon[32710]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-25T12:05:05.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:05 smithi169 ceph-mon[32710]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-25T12:05:05.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:05 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2817017125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:05.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:05 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] up:active 2024-04-25T12:05:05.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:05 smithi169 ceph-mon[32710]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T12:05:05.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:05 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2498485123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:05.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:05 smithi150 ceph-mon[32250]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T12:05:05.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:05 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/354208802' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T12:05:05.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:05 smithi150 ceph-mon[32250]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-25T12:05:05.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:05 smithi150 ceph-mon[32250]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-25T12:05:05.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:05 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2817017125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:05.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:05 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] up:active 2024-04-25T12:05:05.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:05 smithi150 ceph-mon[32250]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T12:05:05.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:05 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2498485123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:06.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:05 smithi019 ceph-mon[29213]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T12:05:06.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:05 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/354208802' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T12:05:06.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:05 smithi019 ceph-mon[29213]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-25T12:05:06.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:05 smithi019 ceph-mon[29213]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-25T12:05:06.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:05 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2817017125' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:06.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:05 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] up:active 2024-04-25T12:05:06.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:05 smithi019 ceph-mon[29213]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T12:05:06.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:05 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2498485123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:06.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.677+0000 7f48f37fe640 1 -- 172.21.15.19:0/4065298233 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f48f40bce50 con 0x7f4904100e90 2024-04-25T12:05:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.679+0000 7f4908bcc640 1 -- 172.21.15.19:0/4065298233 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f48ec0a3bb0 msgr2=0x7f48ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.680+0000 7f4908bcc640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f48ec0a3bb0 0x7f48ec0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4904199950 tx=0x7f48f8008040 comp rx=0 tx=0).stop 2024-04-25T12:05:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.680+0000 7f4908bcc640 1 -- 172.21.15.19:0/4065298233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4904100e90 msgr2=0x7f4904198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.680+0000 7f4908bcc640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4904100e90 0x7f4904198440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f48f4031570 tx=0x7f48f4032d80 comp rx=0 tx=0).stop 2024-04-25T12:05:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.680+0000 7f4908bcc640 1 -- 172.21.15.19:0/4065298233 shutdown_connections 2024-04-25T12:05:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.680+0000 7f4908bcc640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f48ec0a3bb0 0x7f48ec0a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.680+0000 7f4908bcc640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f49041029a0 0x7f490419f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.680+0000 7f4908bcc640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4904101fe0 0x7f4904198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.680+0000 7f4908bcc640 1 --2- 172.21.15.19:0/4065298233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4904100e90 0x7f4904198440 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.680+0000 7f4908bcc640 1 -- 172.21.15.19:0/4065298233 >> 172.21.15.19:0/4065298233 conn(0x7f49040fc400 msgr2=0x7f4904105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.680+0000 7f4908bcc640 1 -- 172.21.15.19:0/4065298233 shutdown_connections 2024-04-25T12:05:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.680+0000 7f4908bcc640 1 -- 172.21.15.19:0/4065298233 wait complete. 2024-04-25T12:05:06.691 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-25T12:05:06.692 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-25T12:05:06.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.807+0000 7fd89b5ed640 1 -- 172.21.15.19:0/487735948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd894103b60 msgr2=0x7fd89410a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:06.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.807+0000 7fd89b5ed640 1 --2- 172.21.15.19:0/487735948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd894103b60 0x7fd89410a3f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009900 tx=0x7fd88c02a410 comp rx=0 tx=0).stop 2024-04-25T12:05:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.807+0000 7fd89b5ed640 1 -- 172.21.15.19:0/487735948 shutdown_connections 2024-04-25T12:05:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.807+0000 7fd89b5ed640 1 --2- 172.21.15.19:0/487735948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd894103b60 0x7fd89410a3f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.807+0000 7fd89b5ed640 1 --2- 172.21.15.19:0/487735948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8941031a0 0x7fd894103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.807+0000 7fd89b5ed640 1 --2- 172.21.15.19:0/487735948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd894101fb0 0x7fd8941023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.807+0000 7fd89b5ed640 1 -- 172.21.15.19:0/487735948 >> 172.21.15.19:0/487735948 conn(0x7fd8940fd520 msgr2=0x7fd8940ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.807+0000 7fd89b5ed640 1 -- 172.21.15.19:0/487735948 shutdown_connections 2024-04-25T12:05:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.807+0000 7fd89b5ed640 1 -- 172.21.15.19:0/487735948 wait complete. 2024-04-25T12:05:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.807+0000 7fd89b5ed640 1 Processor -- start 2024-04-25T12:05:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd89b5ed640 1 -- start start 2024-04-25T12:05:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd89b5ed640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd894101fb0 0x7fd894075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd89b5ed640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8941031a0 0x7fd894074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd89b5ed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd894103b60 0x7fd894074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd89b5ed640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd89410dc70 con 0x7fd894103b60 2024-04-25T12:05:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd89b5ed640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd89410daf0 con 0x7fd8941031a0 2024-04-25T12:05:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd89b5ed640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd89410ddf0 con 0x7fd894101fb0 2024-04-25T12:05:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd899b8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd894103b60 0x7fd894074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd899b8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd894103b60 0x7fd894074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38966/0 (socket says 172.21.15.19:38966) 2024-04-25T12:05:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd899b8a640 1 -- 172.21.15.19:0/1338804289 learned_addr learned my addr 172.21.15.19:0/1338804289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd899389640 1 --2- 172.21.15.19:0/1338804289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd894101fb0 0x7fd894075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd899b8a640 1 -- 172.21.15.19:0/1338804289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd894101fb0 msgr2=0x7fd894075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd898b88640 1 --2- 172.21.15.19:0/1338804289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8941031a0 0x7fd894074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd899b8a640 1 --2- 172.21.15.19:0/1338804289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd894101fb0 0x7fd894075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd899b8a640 1 -- 172.21.15.19:0/1338804289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8941031a0 msgr2=0x7fd894074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd899b8a640 1 --2- 172.21.15.19:0/1338804289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8941031a0 0x7fd894074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd899b8a640 1 -- 172.21.15.19:0/1338804289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd894075110 con 0x7fd894103b60 2024-04-25T12:05:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd899b8a640 1 --2- 172.21.15.19:0/1338804289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd894103b60 0x7fd894074880 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd88c005940 tx=0x7fd88c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.808+0000 7fd88a7fc640 1 -- 172.21.15.19:0/1338804289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c03f030 con 0x7fd894103b60 2024-04-25T12:05:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.809+0000 7fd88a7fc640 1 -- 172.21.15.19:0/1338804289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd88c003cd0 con 0x7fd894103b60 2024-04-25T12:05:06.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.809+0000 7fd89b5ed640 1 -- 172.21.15.19:0/1338804289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8941a2380 con 0x7fd894103b60 2024-04-25T12:05:06.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.809+0000 7fd88a7fc640 1 -- 172.21.15.19:0/1338804289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c035070 con 0x7fd894103b60 2024-04-25T12:05:06.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.809+0000 7fd89b5ed640 1 -- 172.21.15.19:0/1338804289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8941a2950 con 0x7fd894103b60 2024-04-25T12:05:06.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.810+0000 7fd89b5ed640 1 -- 172.21.15.19:0/1338804289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd894109320 con 0x7fd894103b60 2024-04-25T12:05:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.811+0000 7fd88a7fc640 1 -- 172.21.15.19:0/1338804289 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd88c042030 con 0x7fd894103b60 2024-04-25T12:05:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.811+0000 7fd88a7fc640 1 --2- 172.21.15.19:0/1338804289 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd8800a3bb0 0x7fd8800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.811+0000 7fd88a7fc640 1 -- 172.21.15.19:0/1338804289 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd88c0f0c40 con 0x7fd894103b60 2024-04-25T12:05:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.811+0000 7fd899389640 1 --2- 172.21.15.19:0/1338804289 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd8800a3bb0 0x7fd8800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.812+0000 7fd899389640 1 --2- 172.21.15.19:0/1338804289 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd8800a3bb0 0x7fd8800a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd894103000 tx=0x7fd890008230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:06.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:06.815+0000 7fd88a7fc640 1 -- 172.21.15.19:0/1338804289 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd88c0bd770 con 0x7fd894103b60 2024-04-25T12:05:06.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:06 smithi169 ceph-mon[32710]: pgmap v289: 97 pgs: 12 active+clean, 14 creating+peering, 71 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:05:06.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:06 smithi169 ceph-mon[32710]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T12:05:06.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:06 smithi169 ceph-mon[32710]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T12:05:06.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:06 smithi169 ceph-mon[32710]: Cluster is now healthy 2024-04-25T12:05:06.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:06 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/4065298233' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T12:05:06.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:06 smithi150 ceph-mon[32250]: pgmap v289: 97 pgs: 12 active+clean, 14 creating+peering, 71 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:05:06.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:06 smithi150 ceph-mon[32250]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T12:05:06.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:06 smithi150 ceph-mon[32250]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T12:05:06.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:06 smithi150 ceph-mon[32250]: Cluster is now healthy 2024-04-25T12:05:06.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:06 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/4065298233' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T12:05:07.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.045+0000 7fd89b5ed640 1 -- 172.21.15.19:0/1338804289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fd89402d5c0 con 0x7fd894103b60 2024-04-25T12:05:07.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:06 smithi019 ceph-mon[29213]: pgmap v289: 97 pgs: 12 active+clean, 14 creating+peering, 71 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:05:07.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:06 smithi019 ceph-mon[29213]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T12:05:07.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:06 smithi019 ceph-mon[29213]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T12:05:07.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:06 smithi019 ceph-mon[29213]: Cluster is now healthy 2024-04-25T12:05:07.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:06 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/4065298233' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T12:05:07.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.684+0000 7fd88a7fc640 1 -- 172.21.15.19:0/1338804289 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7fd88c0bd070 con 0x7fd894103b60 2024-04-25T12:05:07.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.688+0000 7fd89b5ed640 1 -- 172.21.15.19:0/1338804289 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd8800a3bb0 msgr2=0x7fd8800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:07.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.688+0000 7fd89b5ed640 1 --2- 172.21.15.19:0/1338804289 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd8800a3bb0 0x7fd8800a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd894103000 tx=0x7fd890008230 comp rx=0 tx=0).stop 2024-04-25T12:05:07.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.688+0000 7fd89b5ed640 1 -- 172.21.15.19:0/1338804289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd894103b60 msgr2=0x7fd894074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:07.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.688+0000 7fd89b5ed640 1 --2- 172.21.15.19:0/1338804289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd894103b60 0x7fd894074880 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd88c005940 tx=0x7fd88c0043b0 comp rx=0 tx=0).stop 2024-04-25T12:05:07.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.688+0000 7fd89b5ed640 1 -- 172.21.15.19:0/1338804289 shutdown_connections 2024-04-25T12:05:07.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.688+0000 7fd89b5ed640 1 --2- 172.21.15.19:0/1338804289 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd8800a3bb0 0x7fd8800a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:07.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.688+0000 7fd89b5ed640 1 --2- 172.21.15.19:0/1338804289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd894103b60 0x7fd894074880 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:07.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.688+0000 7fd89b5ed640 1 --2- 172.21.15.19:0/1338804289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8941031a0 0x7fd894074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:07.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.688+0000 7fd89b5ed640 1 --2- 172.21.15.19:0/1338804289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd894101fb0 0x7fd894075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:07.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.688+0000 7fd89b5ed640 1 -- 172.21.15.19:0/1338804289 >> 172.21.15.19:0/1338804289 conn(0x7fd8940fd520 msgr2=0x7fd8941064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:07.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.688+0000 7fd89b5ed640 1 -- 172.21.15.19:0/1338804289 shutdown_connections 2024-04-25T12:05:07.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.688+0000 7fd89b5ed640 1 -- 172.21.15.19:0/1338804289 wait complete. 2024-04-25T12:05:07.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T12:05:07.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.816+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/3623913559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d94100e90 msgr2=0x7f5d941012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:07.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.816+0000 7f5d9b1ed640 1 --2- 172.21.15.19:0/3623913559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d94100e90 0x7f5d941012b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009900 tx=0x7f5d8802a410 comp rx=0 tx=0).stop 2024-04-25T12:05:07.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/3623913559 shutdown_connections 2024-04-25T12:05:07.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 --2- 172.21.15.19:0/3623913559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d941029a0 0x7f5d941092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:07.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 --2- 172.21.15.19:0/3623913559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d94101fe0 0x7f5d94102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:07.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 --2- 172.21.15.19:0/3623913559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d94100e90 0x7f5d941012b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:07.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/3623913559 >> 172.21.15.19:0/3623913559 conn(0x7f5d940fc400 msgr2=0x7f5d940fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:07.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/3623913559 shutdown_connections 2024-04-25T12:05:07.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/3623913559 wait complete. 2024-04-25T12:05:07.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 Processor -- start 2024-04-25T12:05:07.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 -- start start 2024-04-25T12:05:07.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d94100e90 0x7f5d94069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:07.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d94101fe0 0x7f5d9419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:07.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d941029a0 0x7f5d9419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:07.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9410cb20 con 0x7f5d94101fe0 2024-04-25T12:05:07.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9410c9a0 con 0x7f5d941029a0 2024-04-25T12:05:07.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d9b1ed640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9410cca0 con 0x7f5d94100e90 2024-04-25T12:05:07.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.817+0000 7f5d93fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d94101fe0 0x7f5d9419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:07.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d93fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d94101fe0 0x7f5d9419cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38970/0 (socket says 172.21.15.19:38970) 2024-04-25T12:05:07.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d93fff640 1 -- 172.21.15.19:0/93385501 learned_addr learned my addr 172.21.15.19:0/93385501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:07.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d98f89640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d94100e90 0x7f5d94069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d9978a640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d941029a0 0x7f5d9419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d93fff640 1 -- 172.21.15.19:0/93385501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d94100e90 msgr2=0x7f5d94069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d93fff640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d94100e90 0x7f5d94069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d93fff640 1 -- 172.21.15.19:0/93385501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d941029a0 msgr2=0x7f5d9419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d93fff640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d941029a0 0x7f5d9419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d93fff640 1 -- 172.21.15.19:0/93385501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9419fec0 con 0x7f5d94101fe0 2024-04-25T12:05:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d9978a640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d941029a0 0x7f5d9419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:05:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d93fff640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d94101fe0 0x7f5d9419cfc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000e9c0 tx=0x7f5d8000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d91ffb640 1 -- 172.21.15.19:0/93385501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000cd80 con 0x7f5d94101fe0 2024-04-25T12:05:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d91ffb640 1 -- 172.21.15.19:0/93385501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5d800035f0 con 0x7f5d94101fe0 2024-04-25T12:05:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/93385501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d941a01b0 con 0x7f5d94101fe0 2024-04-25T12:05:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d91ffb640 1 -- 172.21.15.19:0/93385501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80010690 con 0x7f5d94101fe0 2024-04-25T12:05:07.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.818+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/93385501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d941a0700 con 0x7f5d94101fe0 2024-04-25T12:05:07.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.819+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/93385501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d941081a0 con 0x7f5d94101fe0 2024-04-25T12:05:07.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.820+0000 7f5d91ffb640 1 -- 172.21.15.19:0/93385501 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5d80003b30 con 0x7f5d94101fe0 2024-04-25T12:05:07.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.821+0000 7f5d91ffb640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5d840a3c30 0x7f5d840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:07.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.821+0000 7f5d91ffb640 1 -- 172.21.15.19:0/93385501 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f5d800d06e0 con 0x7f5d94101fe0 2024-04-25T12:05:07.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.821+0000 7f5d98f89640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5d840a3c30 0x7f5d840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:07.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.821+0000 7f5d98f89640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5d840a3c30 0x7f5d840a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5d88000c00 tx=0x7f5d88031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:07.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:07.825+0000 7f5d91ffb640 1 -- 172.21.15.19:0/93385501 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d8009d170 con 0x7f5d94101fe0 2024-04-25T12:05:08.012 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:07 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/4065298233' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T12:05:08.012 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:07 smithi169 ceph-mon[32710]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T12:05:08.012 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:07 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1338804289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T12:05:08.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:07 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/4065298233' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T12:05:08.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:07 smithi019 ceph-mon[29213]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T12:05:08.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:07 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1338804289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T12:05:08.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.051+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/93385501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f5d94062b50 con 0x7f5d94101fe0 2024-04-25T12:05:08.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:07 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/4065298233' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T12:05:08.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:07 smithi150 ceph-mon[32250]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T12:05:08.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:07 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1338804289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T12:05:08.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.692+0000 7f5d91ffb640 1 -- 172.21.15.19:0/93385501 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f5d8009ca70 con 0x7f5d94101fe0 2024-04-25T12:05:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.695+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/93385501 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5d840a3c30 msgr2=0x7f5d840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.695+0000 7f5d9b1ed640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5d840a3c30 0x7f5d840a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5d88000c00 tx=0x7f5d88031510 comp rx=0 tx=0).stop 2024-04-25T12:05:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.695+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/93385501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d94101fe0 msgr2=0x7f5d9419cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.695+0000 7f5d9b1ed640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d94101fe0 0x7f5d9419cfc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000e9c0 tx=0x7f5d8000ee80 comp rx=0 tx=0).stop 2024-04-25T12:05:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.695+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/93385501 shutdown_connections 2024-04-25T12:05:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.695+0000 7f5d9b1ed640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5d840a3c30 0x7f5d840a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.695+0000 7f5d9b1ed640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d941029a0 0x7f5d9419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.695+0000 7f5d9b1ed640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d94101fe0 0x7f5d9419cfc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.695+0000 7f5d9b1ed640 1 --2- 172.21.15.19:0/93385501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d94100e90 0x7f5d94069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:08.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.695+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/93385501 >> 172.21.15.19:0/93385501 conn(0x7f5d940fc400 msgr2=0x7f5d94105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:08.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.695+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/93385501 shutdown_connections 2024-04-25T12:05:08.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.695+0000 7f5d9b1ed640 1 -- 172.21.15.19:0/93385501 wait complete. 2024-04-25T12:05:08.710 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T12:05:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.823+0000 7f7388e20640 1 -- 172.21.15.19:0/4102335220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73841053a0 msgr2=0x7f7384107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.823+0000 7f7388e20640 1 --2- 172.21.15.19:0/4102335220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73841053a0 0x7f7384107790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f736c009900 tx=0x7f736c02a410 comp rx=0 tx=0).stop 2024-04-25T12:05:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.823+0000 7f7388e20640 1 -- 172.21.15.19:0/4102335220 shutdown_connections 2024-04-25T12:05:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.823+0000 7f7388e20640 1 --2- 172.21.15.19:0/4102335220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7384107cd0 0x7f738410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.823+0000 7f7388e20640 1 --2- 172.21.15.19:0/4102335220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73841053a0 0x7f7384107790 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.823+0000 7f7388e20640 1 --2- 172.21.15.19:0/4102335220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7384102a70 0x7f7384104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:08.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.823+0000 7f7388e20640 1 -- 172.21.15.19:0/4102335220 >> 172.21.15.19:0/4102335220 conn(0x7f73840fc400 msgr2=0x7f73840fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.824+0000 7f7388e20640 1 -- 172.21.15.19:0/4102335220 shutdown_connections 2024-04-25T12:05:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.824+0000 7f7388e20640 1 -- 172.21.15.19:0/4102335220 wait complete. 2024-04-25T12:05:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.824+0000 7f7388e20640 1 Processor -- start 2024-04-25T12:05:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.824+0000 7f7388e20640 1 -- start start 2024-04-25T12:05:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.824+0000 7f7388e20640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7384102a70 0x7f7384198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.824+0000 7f7388e20640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73841053a0 0x7f73841988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.824+0000 7f7388e20640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7384107cd0 0x7f738419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.824+0000 7f7388e20640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f738410dc70 con 0x7f73841053a0 2024-04-25T12:05:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.824+0000 7f7388e20640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f738410daf0 con 0x7f7384102a70 2024-04-25T12:05:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f7388e20640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f738410ddf0 con 0x7f7384107cd0 2024-04-25T12:05:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f737bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73841053a0 0x7f73841988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f737bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73841053a0 0x7f73841988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38976/0 (socket says 172.21.15.19:38976) 2024-04-25T12:05:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f737bfff640 1 -- 172.21.15.19:0/796491516 learned_addr learned my addr 172.21.15.19:0/796491516 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f7382d9d640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7384107cd0 0x7f738419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f738259c640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7384102a70 0x7f7384198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f737bfff640 1 -- 172.21.15.19:0/796491516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7384107cd0 msgr2=0x7f738419f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f737bfff640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7384107cd0 0x7f738419f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f737bfff640 1 -- 172.21.15.19:0/796491516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7384102a70 msgr2=0x7f7384198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f737bfff640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7384102a70 0x7f7384198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f737bfff640 1 -- 172.21.15.19:0/796491516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738419ff90 con 0x7f73841053a0 2024-04-25T12:05:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f738259c640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7384102a70 0x7f7384198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:05:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f737bfff640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73841053a0 0x7f73841988b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f736c006010 tx=0x7f736c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f737b7fe640 1 -- 172.21.15.19:0/796491516 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c03f030 con 0x7f73841053a0 2024-04-25T12:05:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f737b7fe640 1 -- 172.21.15.19:0/796491516 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f736c003cd0 con 0x7f73841053a0 2024-04-25T12:05:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f7388e20640 1 -- 172.21.15.19:0/796491516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73841a0220 con 0x7f73841053a0 2024-04-25T12:05:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f737b7fe640 1 -- 172.21.15.19:0/796491516 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c035070 con 0x7f73841053a0 2024-04-25T12:05:08.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.825+0000 7f7388e20640 1 -- 172.21.15.19:0/796491516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73841a0790 con 0x7f73841053a0 2024-04-25T12:05:08.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.826+0000 7f7388e20640 1 -- 172.21.15.19:0/796491516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73840fdef0 con 0x7f73841053a0 2024-04-25T12:05:08.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.827+0000 7f737b7fe640 1 -- 172.21.15.19:0/796491516 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f736c042030 con 0x7f73841053a0 2024-04-25T12:05:08.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.828+0000 7f737b7fe640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f73700a3bb0 0x7f73700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:08.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.828+0000 7f738259c640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f73700a3bb0 0x7f73700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:08.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.828+0000 7f737b7fe640 1 -- 172.21.15.19:0/796491516 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f736c0f0830 con 0x7f73841053a0 2024-04-25T12:05:08.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.828+0000 7f738259c640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f73700a3bb0 0x7f73700a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7374009b50 tx=0x7f7374005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:08.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:08.832+0000 7f737b7fe640 1 -- 172.21.15.19:0/796491516 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f736c0bd360 con 0x7f73841053a0 2024-04-25T12:05:09.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:08 smithi019 ceph-mon[29213]: pgmap v291: 97 pgs: 53 active+clean, 14 creating+peering, 30 unknown; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 6 op/s 2024-04-25T12:05:09.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:08 smithi019 ceph-mon[29213]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T12:05:09.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:08 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1338804289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T12:05:09.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:08 smithi019 ceph-mon[29213]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T12:05:09.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:08 smithi019 ceph-mon[29213]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T12:05:09.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:08 smithi019 ceph-mon[29213]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-25T12:05:09.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:08 smithi019 ceph-mon[29213]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-25T12:05:09.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:08 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/93385501' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T12:05:09.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.060+0000 7f7388e20640 1 -- 172.21.15.19:0/796491516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f738402d810 con 0x7f73841053a0 2024-04-25T12:05:09.064 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:08 smithi150 ceph-mon[32250]: pgmap v291: 97 pgs: 53 active+clean, 14 creating+peering, 30 unknown; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 6 op/s 2024-04-25T12:05:09.064 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:08 smithi150 ceph-mon[32250]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T12:05:09.064 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:08 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1338804289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T12:05:09.064 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:08 smithi150 ceph-mon[32250]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T12:05:09.064 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:08 smithi150 ceph-mon[32250]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T12:05:09.064 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:08 smithi150 ceph-mon[32250]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-25T12:05:09.064 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:08 smithi150 ceph-mon[32250]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-25T12:05:09.065 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:08 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/93385501' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T12:05:09.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:08 smithi169 ceph-mon[32710]: pgmap v291: 97 pgs: 53 active+clean, 14 creating+peering, 30 unknown; 578 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 6 op/s 2024-04-25T12:05:09.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:08 smithi169 ceph-mon[32710]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T12:05:09.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:08 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1338804289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T12:05:09.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:08 smithi169 ceph-mon[32710]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T12:05:09.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:08 smithi169 ceph-mon[32710]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T12:05:09.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:08 smithi169 ceph-mon[32710]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-25T12:05:09.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:08 smithi169 ceph-mon[32710]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-25T12:05:09.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:08 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/93385501' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T12:05:09.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.705+0000 7f737b7fe640 1 -- 172.21.15.19:0/796491516 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f736c0dbe20 con 0x7f73841053a0 2024-04-25T12:05:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.708+0000 7f7388e20640 1 -- 172.21.15.19:0/796491516 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f73700a3bb0 msgr2=0x7f73700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.708+0000 7f7388e20640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f73700a3bb0 0x7f73700a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7374009b50 tx=0x7f7374005f90 comp rx=0 tx=0).stop 2024-04-25T12:05:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.708+0000 7f7388e20640 1 -- 172.21.15.19:0/796491516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73841053a0 msgr2=0x7f73841988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.708+0000 7f7388e20640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73841053a0 0x7f73841988b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f736c006010 tx=0x7f736c0042f0 comp rx=0 tx=0).stop 2024-04-25T12:05:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.708+0000 7f7388e20640 1 -- 172.21.15.19:0/796491516 shutdown_connections 2024-04-25T12:05:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.708+0000 7f7388e20640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f73700a3bb0 0x7f73700a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:09.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.708+0000 7f7388e20640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7384107cd0 0x7f738419f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.708+0000 7f7388e20640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73841053a0 0x7f73841988b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.708+0000 7f7388e20640 1 --2- 172.21.15.19:0/796491516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7384102a70 0x7f7384198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.708+0000 7f7388e20640 1 -- 172.21.15.19:0/796491516 >> 172.21.15.19:0/796491516 conn(0x7f73840fc400 msgr2=0x7f7384103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.708+0000 7f7388e20640 1 -- 172.21.15.19:0/796491516 shutdown_connections 2024-04-25T12:05:09.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.708+0000 7f7388e20640 1 -- 172.21.15.19:0/796491516 wait complete. 2024-04-25T12:05:09.723 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-25T12:05:09.723 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T12:05:09.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.837+0000 7fa4b7322640 1 -- 172.21.15.19:0/3497896596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4b0100e60 msgr2=0x7fa4b0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:09.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.837+0000 7fa4b7322640 1 --2- 172.21.15.19:0/3497896596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4b0100e60 0x7fa4b0101280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009900 tx=0x7fa4ac02a410 comp rx=0 tx=0).stop 2024-04-25T12:05:09.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.837+0000 7fa4b7322640 1 -- 172.21.15.19:0/3497896596 shutdown_connections 2024-04-25T12:05:09.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.837+0000 7fa4b7322640 1 --2- 172.21.15.19:0/3497896596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b0102a00 0x7fa4b0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:09.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.837+0000 7fa4b7322640 1 --2- 172.21.15.19:0/3497896596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4b0102040 0x7fa4b01024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:09.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.837+0000 7fa4b7322640 1 --2- 172.21.15.19:0/3497896596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4b0100e60 0x7fa4b0101280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:09.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.837+0000 7fa4b7322640 1 -- 172.21.15.19:0/3497896596 >> 172.21.15.19:0/3497896596 conn(0x7fa4b00fc440 msgr2=0x7fa4b00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:09.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.837+0000 7fa4b7322640 1 -- 172.21.15.19:0/3497896596 shutdown_connections 2024-04-25T12:05:09.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.837+0000 7fa4b7322640 1 -- 172.21.15.19:0/3497896596 wait complete. 2024-04-25T12:05:09.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b7322640 1 Processor -- start 2024-04-25T12:05:09.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b7322640 1 -- start start 2024-04-25T12:05:09.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b7322640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4b0100e60 0x7fa4b0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:09.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b7322640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b0102040 0x7fa4b0198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:09.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b7322640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4b0102a00 0x7fa4b019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:09.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b7322640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b010cb30 con 0x7fa4b0102a00 2024-04-25T12:05:09.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b7322640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b010c9b0 con 0x7fa4b0102040 2024-04-25T12:05:09.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b7322640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b010ccb0 con 0x7fa4b0100e60 2024-04-25T12:05:09.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b58bf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4b0102a00 0x7fa4b019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:09.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b58bf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4b0102a00 0x7fa4b019f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38996/0 (socket says 172.21.15.19:38996) 2024-04-25T12:05:09.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b58bf640 1 -- 172.21.15.19:0/1460449447 learned_addr learned my addr 172.21.15.19:0/1460449447 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:09.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b50be640 1 --2- 172.21.15.19:0/1460449447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4b0100e60 0x7fa4b0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:09.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b48bd640 1 --2- 172.21.15.19:0/1460449447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b0102040 0x7fa4b0198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:09.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b58bf640 1 -- 172.21.15.19:0/1460449447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4b0100e60 msgr2=0x7fa4b0198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:09.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b58bf640 1 --2- 172.21.15.19:0/1460449447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4b0100e60 0x7fa4b0198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:09.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b58bf640 1 -- 172.21.15.19:0/1460449447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b0102040 msgr2=0x7fa4b0198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:09.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b58bf640 1 --2- 172.21.15.19:0/1460449447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b0102040 0x7fa4b0198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:09.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.838+0000 7fa4b58bf640 1 -- 172.21.15.19:0/1460449447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b019ff20 con 0x7fa4b0102a00 2024-04-25T12:05:09.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.839+0000 7fa4b58bf640 1 --2- 172.21.15.19:0/1460449447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4b0102a00 0x7fa4b019f8b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009fd0 tx=0x7fa4a800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:09.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.839+0000 7fa4a67fc640 1 -- 172.21.15.19:0/1460449447 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8017070 con 0x7fa4b0102a00 2024-04-25T12:05:09.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.839+0000 7fa4a67fc640 1 -- 172.21.15.19:0/1460449447 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa4a8003e40 con 0x7fa4b0102a00 2024-04-25T12:05:09.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.839+0000 7fa4b7322640 1 -- 172.21.15.19:0/1460449447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b01a0210 con 0x7fa4b0102a00 2024-04-25T12:05:09.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.839+0000 7fa4a67fc640 1 -- 172.21.15.19:0/1460449447 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8004820 con 0x7fa4b0102a00 2024-04-25T12:05:09.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.839+0000 7fa4b7322640 1 -- 172.21.15.19:0/1460449447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b01a0710 con 0x7fa4b0102a00 2024-04-25T12:05:09.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.840+0000 7fa4b7322640 1 -- 172.21.15.19:0/1460449447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b01a0970 con 0x7fa4b0102a00 2024-04-25T12:05:09.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.841+0000 7fa4a67fc640 1 -- 172.21.15.19:0/1460449447 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa4a8003970 con 0x7fa4b0102a00 2024-04-25T12:05:09.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.841+0000 7fa4a67fc640 1 --2- 172.21.15.19:0/1460449447 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:09.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.842+0000 7fa4b50be640 1 --2- 172.21.15.19:0/1460449447 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:09.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.842+0000 7fa4a67fc640 1 -- 172.21.15.19:0/1460449447 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa4a80d0110 con 0x7fa4b0102a00 2024-04-25T12:05:09.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.842+0000 7fa4b50be640 1 --2- 172.21.15.19:0/1460449447 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009230 tx=0x7fa4ac031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:09.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:09.846+0000 7fa4a67fc640 1 -- 172.21.15.19:0/1460449447 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4a8003c60 con 0x7fa4b0102a00 2024-04-25T12:05:09.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:09 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/93385501' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T12:05:09.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:09 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] up:active 2024-04-25T12:05:09.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:09 smithi169 ceph-mon[32710]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T12:05:09.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:09 smithi169 ceph-mon[32710]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-25T12:05:09.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:09 smithi169 ceph-mon[32710]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-25T12:05:09.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:09 smithi169 ceph-mon[32710]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T12:05:09.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:09 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/796491516' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T12:05:09.906 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:05:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i[64470]: 2024-04-25T12:05:09.556+0000 7faf51719700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T12:05:09.906 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:05:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i[64470]: 2024-04-25T12:05:09.556+0000 7faf51719700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T12:05:10.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:09 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/93385501' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T12:05:10.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:09 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] up:active 2024-04-25T12:05:10.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:09 smithi019 ceph-mon[29213]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T12:05:10.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:09 smithi019 ceph-mon[29213]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-25T12:05:10.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:09 smithi019 ceph-mon[29213]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-25T12:05:10.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:09 smithi019 ceph-mon[29213]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T12:05:10.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:09 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/796491516' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T12:05:10.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.061+0000 7fa4b7322640 1 -- 172.21.15.19:0/1460449447 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa4b006ac60 con 0x7fa49c0a3bb0 2024-04-25T12:05:10.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.083+0000 7fa4a67fc640 1 -- 172.21.15.19:0/1460449447 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa4b006ac60 con 0x7fa49c0a3bb0 2024-04-25T12:05:10.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.085+0000 7fa4b7322640 1 -- 172.21.15.19:0/1460449447 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa49c0a3bb0 msgr2=0x7fa49c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.085+0000 7fa4b7322640 1 --2- 172.21.15.19:0/1460449447 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009230 tx=0x7fa4ac031510 comp rx=0 tx=0).stop 2024-04-25T12:05:10.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.085+0000 7fa4b7322640 1 -- 172.21.15.19:0/1460449447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4b0102a00 msgr2=0x7fa4b019f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.085+0000 7fa4b7322640 1 --2- 172.21.15.19:0/1460449447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4b0102a00 0x7fa4b019f8b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009fd0 tx=0x7fa4a800b260 comp rx=0 tx=0).stop 2024-04-25T12:05:10.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.085+0000 7fa4b7322640 1 -- 172.21.15.19:0/1460449447 shutdown_connections 2024-04-25T12:05:10.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.085+0000 7fa4b7322640 1 --2- 172.21.15.19:0/1460449447 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.085+0000 7fa4b7322640 1 --2- 172.21.15.19:0/1460449447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4b0102a00 0x7fa4b019f8b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.085+0000 7fa4b7322640 1 --2- 172.21.15.19:0/1460449447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4b0102040 0x7fa4b0198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.085+0000 7fa4b7322640 1 --2- 172.21.15.19:0/1460449447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4b0100e60 0x7fa4b0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.085+0000 7fa4b7322640 1 -- 172.21.15.19:0/1460449447 >> 172.21.15.19:0/1460449447 conn(0x7fa4b00fc440 msgr2=0x7fa4b006a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:10.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.085+0000 7fa4b7322640 1 -- 172.21.15.19:0/1460449447 shutdown_connections 2024-04-25T12:05:10.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.085+0000 7fa4b7322640 1 -- 172.21.15.19:0/1460449447 wait complete. 2024-04-25T12:05:10.093 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T12:05:10.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:09 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/93385501' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T12:05:10.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:09 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] up:active 2024-04-25T12:05:10.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:09 smithi150 ceph-mon[32250]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T12:05:10.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:09 smithi150 ceph-mon[32250]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-25T12:05:10.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:09 smithi150 ceph-mon[32250]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-25T12:05:10.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:09 smithi150 ceph-mon[32250]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T12:05:10.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:09 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/796491516' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T12:05:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.210+0000 7f2528459640 1 -- 172.21.15.19:0/793568904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2520100e90 msgr2=0x7f25201012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.210+0000 7f2528459640 1 --2- 172.21.15.19:0/793568904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2520100e90 0x7f25201012b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2518009900 tx=0x7f251802a410 comp rx=0 tx=0).stop 2024-04-25T12:05:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.210+0000 7f2528459640 1 -- 172.21.15.19:0/793568904 shutdown_connections 2024-04-25T12:05:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.210+0000 7f2528459640 1 --2- 172.21.15.19:0/793568904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25201029a0 0x7f25201092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.210+0000 7f2528459640 1 --2- 172.21.15.19:0/793568904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2520101fe0 0x7f2520102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.210+0000 7f2528459640 1 --2- 172.21.15.19:0/793568904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2520100e90 0x7f25201012b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.210+0000 7f2528459640 1 -- 172.21.15.19:0/793568904 >> 172.21.15.19:0/793568904 conn(0x7f25200fc400 msgr2=0x7f25200fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:10.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f2528459640 1 -- 172.21.15.19:0/793568904 shutdown_connections 2024-04-25T12:05:10.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f2528459640 1 -- 172.21.15.19:0/793568904 wait complete. 2024-04-25T12:05:10.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f2528459640 1 Processor -- start 2024-04-25T12:05:10.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f2528459640 1 -- start start 2024-04-25T12:05:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f2528459640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2520100e90 0x7f2520198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f2528459640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2520101fe0 0x7f2520198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f2528459640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25201029a0 0x7f252019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f2528459640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f252010dc10 con 0x7f25201029a0 2024-04-25T12:05:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f2528459640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f252010da90 con 0x7f2520101fe0 2024-04-25T12:05:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f2528459640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f252010dd90 con 0x7f2520100e90 2024-04-25T12:05:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f25269f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25201029a0 0x7f252019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f25269f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25201029a0 0x7f252019f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39026/0 (socket says 172.21.15.19:39026) 2024-04-25T12:05:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f25269f6640 1 -- 172.21.15.19:0/1038693830 learned_addr learned my addr 172.21.15.19:0/1038693830 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:10.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f25261f5640 1 --2- 172.21.15.19:0/1038693830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2520100e90 0x7f2520198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:10.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.211+0000 7f25259f4640 1 --2- 172.21.15.19:0/1038693830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2520101fe0 0x7f2520198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:10.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.212+0000 7f25269f6640 1 -- 172.21.15.19:0/1038693830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2520100e90 msgr2=0x7f2520198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.212+0000 7f25269f6640 1 --2- 172.21.15.19:0/1038693830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2520100e90 0x7f2520198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.212+0000 7f25269f6640 1 -- 172.21.15.19:0/1038693830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2520101fe0 msgr2=0x7f2520198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.212+0000 7f25269f6640 1 --2- 172.21.15.19:0/1038693830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2520101fe0 0x7f2520198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.212+0000 7f25269f6640 1 -- 172.21.15.19:0/1038693830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252019ff30 con 0x7f25201029a0 2024-04-25T12:05:10.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.212+0000 7f25269f6640 1 --2- 172.21.15.19:0/1038693830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25201029a0 0x7f252019f8c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f251000efd0 tx=0x7f251000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:10.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.212+0000 7f251f7fe640 1 -- 172.21.15.19:0/1038693830 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510003b70 con 0x7f25201029a0 2024-04-25T12:05:10.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.212+0000 7f251f7fe640 1 -- 172.21.15.19:0/1038693830 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2510003d10 con 0x7f25201029a0 2024-04-25T12:05:10.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.212+0000 7f251f7fe640 1 -- 172.21.15.19:0/1038693830 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510010990 con 0x7f25201029a0 2024-04-25T12:05:10.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.212+0000 7f2528459640 1 -- 172.21.15.19:0/1038693830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25201a0220 con 0x7f25201029a0 2024-04-25T12:05:10.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.212+0000 7f2528459640 1 -- 172.21.15.19:0/1038693830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25201a0720 con 0x7f25201029a0 2024-04-25T12:05:10.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.213+0000 7f2528459640 1 -- 172.21.15.19:0/1038693830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25201081f0 con 0x7f25201029a0 2024-04-25T12:05:10.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.214+0000 7f251f7fe640 1 -- 172.21.15.19:0/1038693830 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2510004020 con 0x7f25201029a0 2024-04-25T12:05:10.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.215+0000 7f251f7fe640 1 --2- 172.21.15.19:0/1038693830 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f25080a3bb0 0x7f25080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:10.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.215+0000 7f25261f5640 1 --2- 172.21.15.19:0/1038693830 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f25080a3bb0 0x7f25080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.215+0000 7f251f7fe640 1 -- 172.21.15.19:0/1038693830 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f25100d0680 con 0x7f25201029a0 2024-04-25T12:05:10.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.215+0000 7f25261f5640 1 --2- 172.21.15.19:0/1038693830 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f25080a3bb0 0x7f25080a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2518000c00 tx=0x7f2518031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:10.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.219+0000 7f251f7fe640 1 -- 172.21.15.19:0/1038693830 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f251009d1b0 con 0x7f25201029a0 2024-04-25T12:05:10.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.432+0000 7f2528459640 1 -- 172.21.15.19:0/1038693830 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2520062390 con 0x7f25080a3bb0 2024-04-25T12:05:10.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.457+0000 7f251f7fe640 1 -- 172.21.15.19:0/1038693830 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2520062390 con 0x7f25080a3bb0 2024-04-25T12:05:10.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.459+0000 7f2528459640 1 -- 172.21.15.19:0/1038693830 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f25080a3bb0 msgr2=0x7f25080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.459+0000 7f2528459640 1 --2- 172.21.15.19:0/1038693830 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f25080a3bb0 0x7f25080a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2518000c00 tx=0x7f2518031510 comp rx=0 tx=0).stop 2024-04-25T12:05:10.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.459+0000 7f2528459640 1 -- 172.21.15.19:0/1038693830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25201029a0 msgr2=0x7f252019f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.459+0000 7f2528459640 1 --2- 172.21.15.19:0/1038693830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25201029a0 0x7f252019f8c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f251000efd0 tx=0x7f251000c6a0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.459+0000 7f2528459640 1 -- 172.21.15.19:0/1038693830 shutdown_connections 2024-04-25T12:05:10.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.459+0000 7f2528459640 1 --2- 172.21.15.19:0/1038693830 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f25080a3bb0 0x7f25080a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.459+0000 7f2528459640 1 --2- 172.21.15.19:0/1038693830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25201029a0 0x7f252019f8c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.459+0000 7f2528459640 1 --2- 172.21.15.19:0/1038693830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2520101fe0 0x7f2520198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.459+0000 7f2528459640 1 --2- 172.21.15.19:0/1038693830 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2520100e90 0x7f2520198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.459+0000 7f2528459640 1 -- 172.21.15.19:0/1038693830 >> 172.21.15.19:0/1038693830 conn(0x7f25200fc400 msgr2=0x7f25201053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:10.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.459+0000 7f2528459640 1 -- 172.21.15.19:0/1038693830 shutdown_connections 2024-04-25T12:05:10.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.459+0000 7f2528459640 1 -- 172.21.15.19:0/1038693830 wait complete. 2024-04-25T12:05:10.466 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:05:10.066+0000 7efe2e03c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:05:10.466 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:05:10.066+0000 7efe2e03c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:05:10.467 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:05:10.066+0000 7efe2e03c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:05:10.467 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:05:10.066+0000 7efe2e03c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:05:10.467 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:05:10.066+0000 7efe2e03c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:05:10.468 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T12:05:10.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.578+0000 7f10f7ce1640 1 -- 172.21.15.19:0/3203874293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f01029a0 msgr2=0x7f10f01092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.578+0000 7f10f7ce1640 1 --2- 172.21.15.19:0/3203874293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f01029a0 0x7f10f01092c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009ca0 tx=0x7f10e802a8c0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.578+0000 7f10f7ce1640 1 -- 172.21.15.19:0/3203874293 shutdown_connections 2024-04-25T12:05:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.578+0000 7f10f7ce1640 1 --2- 172.21.15.19:0/3203874293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f01029a0 0x7f10f01092c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.578+0000 7f10f7ce1640 1 --2- 172.21.15.19:0/3203874293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f0101fe0 0x7f10f0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.578+0000 7f10f7ce1640 1 --2- 172.21.15.19:0/3203874293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10f0100e90 0x7f10f01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.578+0000 7f10f7ce1640 1 -- 172.21.15.19:0/3203874293 >> 172.21.15.19:0/3203874293 conn(0x7f10f00fc400 msgr2=0x7f10f00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.578+0000 7f10f7ce1640 1 -- 172.21.15.19:0/3203874293 shutdown_connections 2024-04-25T12:05:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.578+0000 7f10f7ce1640 1 -- 172.21.15.19:0/3203874293 wait complete. 2024-04-25T12:05:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.578+0000 7f10f7ce1640 1 Processor -- start 2024-04-25T12:05:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.578+0000 7f10f7ce1640 1 -- start start 2024-04-25T12:05:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f7ce1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f0100e90 0x7f10f01982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f7ce1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f0101fe0 0x7f10f0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:10.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f7ce1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10f01029a0 0x7f10f019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f7ce1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10f010dc20 con 0x7f10f0101fe0 2024-04-25T12:05:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f7ce1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f10f010daa0 con 0x7f10f0100e90 2024-04-25T12:05:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f7ce1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f10f010dda0 con 0x7f10f01029a0 2024-04-25T12:05:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f527c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f0101fe0 0x7f10f0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f527c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f0101fe0 0x7f10f0198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39042/0 (socket says 172.21.15.19:39042) 2024-04-25T12:05:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f527c640 1 -- 172.21.15.19:0/885252891 learned_addr learned my addr 172.21.15.19:0/885252891 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f627e640 1 --2- 172.21.15.19:0/885252891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10f01029a0 0x7f10f019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f5a7d640 1 --2- 172.21.15.19:0/885252891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f0100e90 0x7f10f01982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f527c640 1 -- 172.21.15.19:0/885252891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10f01029a0 msgr2=0x7f10f019f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f527c640 1 --2- 172.21.15.19:0/885252891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10f01029a0 0x7f10f019f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f527c640 1 -- 172.21.15.19:0/885252891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f0100e90 msgr2=0x7f10f01982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f527c640 1 --2- 172.21.15.19:0/885252891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f0100e90 0x7f10f01982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f527c640 1 -- 172.21.15.19:0/885252891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f019fef0 con 0x7f10f0101fe0 2024-04-25T12:05:10.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f527c640 1 --2- 172.21.15.19:0/885252891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f0101fe0 0x7f10f0198810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00d770 tx=0x7f10ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10e6ffd640 1 -- 172.21.15.19:0/885252891 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec017070 con 0x7f10f0101fe0 2024-04-25T12:05:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10e6ffd640 1 -- 172.21.15.19:0/885252891 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f10ec009c30 con 0x7f10f0101fe0 2024-04-25T12:05:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f7ce1640 1 -- 172.21.15.19:0/885252891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f01a01e0 con 0x7f10f0101fe0 2024-04-25T12:05:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10e6ffd640 1 -- 172.21.15.19:0/885252891 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec003d70 con 0x7f10f0101fe0 2024-04-25T12:05:10.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.579+0000 7f10f7ce1640 1 -- 172.21.15.19:0/885252891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f01a0780 con 0x7f10f0101fe0 2024-04-25T12:05:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.581+0000 7f10f7ce1640 1 -- 172.21.15.19:0/885252891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f0108140 con 0x7f10f0101fe0 2024-04-25T12:05:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.582+0000 7f10e6ffd640 1 -- 172.21.15.19:0/885252891 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f10ec003f10 con 0x7f10f0101fe0 2024-04-25T12:05:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.582+0000 7f10e6ffd640 1 --2- 172.21.15.19:0/885252891 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f10dc0a3c30 0x7f10dc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.582+0000 7f10f5a7d640 1 --2- 172.21.15.19:0/885252891 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f10dc0a3c30 0x7f10dc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.582+0000 7f10e6ffd640 1 -- 172.21.15.19:0/885252891 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f10ec0cfb10 con 0x7f10f0101fe0 2024-04-25T12:05:10.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.583+0000 7f10f5a7d640 1 --2- 172.21.15.19:0/885252891 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f10dc0a3c30 0x7f10dc0a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009bd0 tx=0x7f10e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:10.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.587+0000 7f10e6ffd640 1 -- 172.21.15.19:0/885252891 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10ec009de0 con 0x7f10f0101fe0 2024-04-25T12:05:10.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-mon[29213]: pgmap v292: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2024-04-25T12:05:10.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/796491516' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T12:05:10.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] up:active 2024-04-25T12:05:10.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-mon[29213]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T12:05:10.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-mon[29213]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-04-25T12:05:10.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-mon[29213]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-04-25T12:05:10.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-mon[29213]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-04-25T12:05:10.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] up:active 2024-04-25T12:05:10.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-mon[29213]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T12:05:10.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-mon[29213]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T12:05:10.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-mon[29213]: Cluster is now healthy 2024-04-25T12:05:10.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-mon[29213]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2024-04-25T12:05:10.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-mon[29213]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-25T12:05:10.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:10 smithi019 ceph-mon[29213]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-25T12:05:10.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.805+0000 7f10f7ce1640 1 -- 172.21.15.19:0/885252891 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f10f0062420 con 0x7f10dc0a3c30 2024-04-25T12:05:10.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.810+0000 7f10e6ffd640 1 -- 172.21.15.19:0/885252891 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f10f0062420 con 0x7f10dc0a3c30 2024-04-25T12:05:10.807 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_0/98b1a5bd-c917-4beb-a173-4e2ad5a447a5 2024-04-25T12:05:10.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.811+0000 7f10f7ce1640 1 -- 172.21.15.19:0/885252891 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f10dc0a3c30 msgr2=0x7f10dc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.811+0000 7f10f7ce1640 1 --2- 172.21.15.19:0/885252891 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f10dc0a3c30 0x7f10dc0a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009bd0 tx=0x7f10e0005f90 comp rx=0 tx=0).stop 2024-04-25T12:05:10.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.811+0000 7f10f7ce1640 1 -- 172.21.15.19:0/885252891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f0101fe0 msgr2=0x7f10f0198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.811+0000 7f10f7ce1640 1 --2- 172.21.15.19:0/885252891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f0101fe0 0x7f10f0198810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00d770 tx=0x7f10ec00dc30 comp rx=0 tx=0).stop 2024-04-25T12:05:10.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.811+0000 7f10f7ce1640 1 -- 172.21.15.19:0/885252891 shutdown_connections 2024-04-25T12:05:10.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.811+0000 7f10f7ce1640 1 --2- 172.21.15.19:0/885252891 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f10dc0a3c30 0x7f10dc0a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.811+0000 7f10f7ce1640 1 --2- 172.21.15.19:0/885252891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f10f01029a0 0x7f10f019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.812+0000 7f10f7ce1640 1 --2- 172.21.15.19:0/885252891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10f0101fe0 0x7f10f0198810 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.812+0000 7f10f7ce1640 1 --2- 172.21.15.19:0/885252891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10f0100e90 0x7f10f01982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.812+0000 7f10f7ce1640 1 -- 172.21.15.19:0/885252891 >> 172.21.15.19:0/885252891 conn(0x7f10f00fc400 msgr2=0x7f10f0105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:10.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.812+0000 7f10f7ce1640 1 -- 172.21.15.19:0/885252891 shutdown_connections 2024-04-25T12:05:10.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.812+0000 7f10f7ce1640 1 -- 172.21.15.19:0/885252891 wait complete. 2024-04-25T12:05:10.820 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T12:05:10.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.929+0000 7f637c2a7640 1 -- 172.21.15.19:0/2872876967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6374102040 msgr2=0x7f63741024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.929+0000 7f637c2a7640 1 --2- 172.21.15.19:0/2872876967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6374102040 0x7f63741024c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6364009900 tx=0x7f636402a410 comp rx=0 tx=0).stop 2024-04-25T12:05:10.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.929+0000 7f637c2a7640 1 -- 172.21.15.19:0/2872876967 shutdown_connections 2024-04-25T12:05:10.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.929+0000 7f637c2a7640 1 --2- 172.21.15.19:0/2872876967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6374102a00 0x7f6374109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.929+0000 7f637c2a7640 1 --2- 172.21.15.19:0/2872876967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6374102040 0x7f63741024c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.929+0000 7f637c2a7640 1 --2- 172.21.15.19:0/2872876967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6374100e60 0x7f6374101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.929+0000 7f637c2a7640 1 -- 172.21.15.19:0/2872876967 >> 172.21.15.19:0/2872876967 conn(0x7f63740fc440 msgr2=0x7f63740fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:10.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.929+0000 7f637c2a7640 1 -- 172.21.15.19:0/2872876967 shutdown_connections 2024-04-25T12:05:10.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.930+0000 7f637c2a7640 1 -- 172.21.15.19:0/2872876967 wait complete. 2024-04-25T12:05:10.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.930+0000 7f637c2a7640 1 Processor -- start 2024-04-25T12:05:10.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.930+0000 7f637c2a7640 1 -- start start 2024-04-25T12:05:10.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.930+0000 7f637c2a7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6374100e60 0x7f637406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:10.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.930+0000 7f637c2a7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6374102040 0x7f637419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:10.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.930+0000 7f637c2a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6374102a00 0x7f637419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:10.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.930+0000 7f637c2a7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f637410cb40 con 0x7f6374102a00 2024-04-25T12:05:10.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.930+0000 7f637c2a7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f637410c9c0 con 0x7f6374102040 2024-04-25T12:05:10.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.930+0000 7f637c2a7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f637410ccc0 con 0x7f6374100e60 2024-04-25T12:05:10.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.930+0000 7f637a844640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6374102a00 0x7f637419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:10.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.930+0000 7f637a844640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6374102a00 0x7f637419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39056/0 (socket says 172.21.15.19:39056) 2024-04-25T12:05:10.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.930+0000 7f637a844640 1 -- 172.21.15.19:0/2168762794 learned_addr learned my addr 172.21.15.19:0/2168762794 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:10.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.930+0000 7f637a043640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6374100e60 0x7f637406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:10.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.930+0000 7f6379842640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6374102040 0x7f637419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:10.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.931+0000 7f637a043640 1 -- 172.21.15.19:0/2168762794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6374102040 msgr2=0x7f637419d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.931+0000 7f637a043640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6374102040 0x7f637419d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.931+0000 7f637a043640 1 -- 172.21.15.19:0/2168762794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6374102a00 msgr2=0x7f637419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:10.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.931+0000 7f637a043640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6374102a00 0x7f637419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:10.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.931+0000 7f637a043640 1 -- 172.21.15.19:0/2168762794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637419ff30 con 0x7f6374100e60 2024-04-25T12:05:10.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.931+0000 7f637a844640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6374102a00 0x7f637419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:05:10.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.931+0000 7f637a043640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6374100e60 0x7f637406b0c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f637000c940 tx=0x7f637000ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:10.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.931+0000 7f636b7fe640 1 -- 172.21.15.19:0/2168762794 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370004220 con 0x7f6374100e60 2024-04-25T12:05:10.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.931+0000 7f637c2a7640 1 -- 172.21.15.19:0/2168762794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63741a01c0 con 0x7f6374100e60 2024-04-25T12:05:10.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.931+0000 7f636b7fe640 1 -- 172.21.15.19:0/2168762794 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f63700043c0 con 0x7f6374100e60 2024-04-25T12:05:10.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.931+0000 7f636b7fe640 1 -- 172.21.15.19:0/2168762794 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637000f450 con 0x7f6374100e60 2024-04-25T12:05:10.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.931+0000 7f637c2a7640 1 -- 172.21.15.19:0/2168762794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63741a0730 con 0x7f6374100e60 2024-04-25T12:05:10.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.932+0000 7f637c2a7640 1 -- 172.21.15.19:0/2168762794 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63740691c0 con 0x7f6374100e60 2024-04-25T12:05:10.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.933+0000 7f636b7fe640 1 -- 172.21.15.19:0/2168762794 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f63700079b0 con 0x7f6374100e60 2024-04-25T12:05:10.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.934+0000 7f636b7fe640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f63600a3bb0 0x7f63600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:10.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.934+0000 7f6379842640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f63600a3bb0 0x7f63600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:10.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.934+0000 7f636b7fe640 1 -- 172.21.15.19:0/2168762794 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f63700d02a0 con 0x7f6374100e60 2024-04-25T12:05:10.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.934+0000 7f6379842640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f63600a3bb0 0x7f63600a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6364000c00 tx=0x7f6364031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:10.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:10.942+0000 7f636b7fe640 1 -- 172.21.15.19:0/2168762794 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6370014030 con 0x7f6374100e60 2024-04-25T12:05:11.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.156+0000 7f637c2a7640 1 -- 172.21.15.19:0/2168762794 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f63740624e0 con 0x7f63600a3bb0 2024-04-25T12:05:11.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:10 smithi169 ceph-mon[32710]: pgmap v292: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2024-04-25T12:05:11.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:10 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/796491516' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T12:05:11.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:10 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] up:active 2024-04-25T12:05:11.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:10 smithi169 ceph-mon[32710]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T12:05:11.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:10 smithi169 ceph-mon[32710]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-04-25T12:05:11.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:10 smithi169 ceph-mon[32710]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-04-25T12:05:11.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:10 smithi169 ceph-mon[32710]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-04-25T12:05:11.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:10 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] up:active 2024-04-25T12:05:11.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:10 smithi169 ceph-mon[32710]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T12:05:11.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:10 smithi169 ceph-mon[32710]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T12:05:11.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:10 smithi169 ceph-mon[32710]: Cluster is now healthy 2024-04-25T12:05:11.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:10 smithi169 ceph-mon[32710]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2024-04-25T12:05:11.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:10 smithi169 ceph-mon[32710]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-25T12:05:11.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:10 smithi169 ceph-mon[32710]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-25T12:05:11.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:10 smithi150 ceph-mon[32250]: pgmap v292: 97 pgs: 97 active+clean; 580 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2024-04-25T12:05:11.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:10 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/796491516' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T12:05:11.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:10 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] up:active 2024-04-25T12:05:11.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:10 smithi150 ceph-mon[32250]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T12:05:11.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:10 smithi150 ceph-mon[32250]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-04-25T12:05:11.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:10 smithi150 ceph-mon[32250]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-04-25T12:05:11.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:10 smithi150 ceph-mon[32250]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-04-25T12:05:11.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:10 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] up:active 2024-04-25T12:05:11.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:10 smithi150 ceph-mon[32250]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T12:05:11.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:10 smithi150 ceph-mon[32250]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T12:05:11.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:10 smithi150 ceph-mon[32250]: Cluster is now healthy 2024-04-25T12:05:11.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:10 smithi150 ceph-mon[32250]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 4 up:standby-replay 4 up:standby 2024-04-25T12:05:11.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:10 smithi150 ceph-mon[32250]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-25T12:05:11.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:10 smithi150 ceph-mon[32250]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-25T12:05:11.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.181+0000 7f636b7fe640 1 -- 172.21.15.19:0/2168762794 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f63740624e0 con 0x7f63600a3bb0 2024-04-25T12:05:11.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.184+0000 7f637c2a7640 1 -- 172.21.15.19:0/2168762794 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f63600a3bb0 msgr2=0x7f63600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:11.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.184+0000 7f637c2a7640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f63600a3bb0 0x7f63600a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6364000c00 tx=0x7f6364031510 comp rx=0 tx=0).stop 2024-04-25T12:05:11.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.184+0000 7f637c2a7640 1 -- 172.21.15.19:0/2168762794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6374100e60 msgr2=0x7f637406b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:11.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.184+0000 7f637c2a7640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6374100e60 0x7f637406b0c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f637000c940 tx=0x7f637000ce00 comp rx=0 tx=0).stop 2024-04-25T12:05:11.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.184+0000 7f637c2a7640 1 -- 172.21.15.19:0/2168762794 shutdown_connections 2024-04-25T12:05:11.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.184+0000 7f637c2a7640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f63600a3bb0 0x7f63600a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.184+0000 7f637c2a7640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6374102a00 0x7f637419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.184+0000 7f637c2a7640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6374102040 0x7f637419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.184+0000 7f637c2a7640 1 --2- 172.21.15.19:0/2168762794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6374100e60 0x7f637406b0c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.184+0000 7f637c2a7640 1 -- 172.21.15.19:0/2168762794 >> 172.21.15.19:0/2168762794 conn(0x7f63740fc440 msgr2=0x7f6374105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:11.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.184+0000 7f637c2a7640 1 -- 172.21.15.19:0/2168762794 shutdown_connections 2024-04-25T12:05:11.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.184+0000 7f637c2a7640 1 -- 172.21.15.19:0/2168762794 wait complete. 2024-04-25T12:05:11.194 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T12:05:11.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.303+0000 7f888e7c6640 1 -- 172.21.15.19:0/2642357101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8888100f10 msgr2=0x7f8888101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:11.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.303+0000 7f888e7c6640 1 --2- 172.21.15.19:0/2642357101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8888100f10 0x7f8888101330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8878009900 tx=0x7f887802a410 comp rx=0 tx=0).stop 2024-04-25T12:05:11.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.303+0000 7f888e7c6640 1 -- 172.21.15.19:0/2642357101 shutdown_connections 2024-04-25T12:05:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.303+0000 7f888e7c6640 1 --2- 172.21.15.19:0/2642357101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8888102a20 0x7f8888109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.303+0000 7f888e7c6640 1 --2- 172.21.15.19:0/2642357101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8888102060 0x7f88881024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.303+0000 7f888e7c6640 1 --2- 172.21.15.19:0/2642357101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8888100f10 0x7f8888101330 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.303+0000 7f888e7c6640 1 -- 172.21.15.19:0/2642357101 >> 172.21.15.19:0/2642357101 conn(0x7f88880fc4a0 msgr2=0x7f88880fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.303+0000 7f888e7c6640 1 -- 172.21.15.19:0/2642357101 shutdown_connections 2024-04-25T12:05:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.303+0000 7f888e7c6640 1 -- 172.21.15.19:0/2642357101 wait complete. 2024-04-25T12:05:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.303+0000 7f888e7c6640 1 Processor -- start 2024-04-25T12:05:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.303+0000 7f888e7c6640 1 -- start start 2024-04-25T12:05:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f888e7c6640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8888100f10 0x7f8888104e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f888e7c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8888102060 0x7f88881034d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f888e7c6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8888102a20 0x7f8888103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f888e7c6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f888810cb10 con 0x7f8888102060 2024-04-25T12:05:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f888e7c6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f888810c990 con 0x7f8888102a20 2024-04-25T12:05:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f888e7c6640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f888810cc90 con 0x7f8888100f10 2024-04-25T12:05:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f88877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8888102060 0x7f88881034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f88877fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8888102060 0x7f88881034d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39080/0 (socket says 172.21.15.19:39080) 2024-04-25T12:05:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f88877fe640 1 -- 172.21.15.19:0/1263096390 learned_addr learned my addr 172.21.15.19:0/1263096390 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f8887fff640 1 --2- 172.21.15.19:0/1263096390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8888100f10 0x7f8888104e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f888cd63640 1 --2- 172.21.15.19:0/1263096390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8888102a20 0x7f8888103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f88877fe640 1 -- 172.21.15.19:0/1263096390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8888100f10 msgr2=0x7f8888104e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f88877fe640 1 --2- 172.21.15.19:0/1263096390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8888100f10 0x7f8888104e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f88877fe640 1 -- 172.21.15.19:0/1263096390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8888102a20 msgr2=0x7f8888103a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f88877fe640 1 --2- 172.21.15.19:0/1263096390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8888102a20 0x7f8888103a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f88877fe640 1 -- 172.21.15.19:0/1263096390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8888104230 con 0x7f8888102060 2024-04-25T12:05:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f88877fe640 1 --2- 172.21.15.19:0/1263096390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8888102060 0x7f88881034d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f887c00d770 tx=0x7f887c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.304+0000 7f88857fa640 1 -- 172.21.15.19:0/1263096390 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c017070 con 0x7f8888102060 2024-04-25T12:05:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.305+0000 7f888e7c6640 1 -- 172.21.15.19:0/1263096390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88881a23d0 con 0x7f8888102060 2024-04-25T12:05:11.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.305+0000 7f88857fa640 1 -- 172.21.15.19:0/1263096390 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f887c009c30 con 0x7f8888102060 2024-04-25T12:05:11.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.305+0000 7f88857fa640 1 -- 172.21.15.19:0/1263096390 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c003d70 con 0x7f8888102060 2024-04-25T12:05:11.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.305+0000 7f888e7c6640 1 -- 172.21.15.19:0/1263096390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88881a2970 con 0x7f8888102060 2024-04-25T12:05:11.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.305+0000 7f888e7c6640 1 -- 172.21.15.19:0/1263096390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88881a2bd0 con 0x7f8888102060 2024-04-25T12:05:11.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.307+0000 7f88857fa640 1 -- 172.21.15.19:0/1263096390 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f887c003f10 con 0x7f8888102060 2024-04-25T12:05:11.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.307+0000 7f88857fa640 1 --2- 172.21.15.19:0/1263096390 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88700a3bb0 0x7f88700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:11.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.307+0000 7f8887fff640 1 --2- 172.21.15.19:0/1263096390 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88700a3bb0 0x7f88700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:11.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.307+0000 7f88857fa640 1 -- 172.21.15.19:0/1263096390 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f887c0cfc20 con 0x7f8888102060 2024-04-25T12:05:11.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.308+0000 7f8887fff640 1 --2- 172.21.15.19:0/1263096390 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88700a3bb0 0x7f88700a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8878000c00 tx=0x7f8878031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.312+0000 7f88857fa640 1 -- 172.21.15.19:0/1263096390 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f887c09c750 con 0x7f8888102060 2024-04-25T12:05:11.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.530+0000 7f888e7c6640 1 -- 172.21.15.19:0/1263096390 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f88880fe660 con 0x7f88700a3bb0 2024-04-25T12:05:11.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.535+0000 7f88857fa640 1 -- 172.21.15.19:0/1263096390 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f88880fe660 con 0x7f88700a3bb0 2024-04-25T12:05:11.533 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_1/19c17715-a266-4b73-9313-334711baa8c6 2024-04-25T12:05:11.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.537+0000 7f888e7c6640 1 -- 172.21.15.19:0/1263096390 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88700a3bb0 msgr2=0x7f88700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:11.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.537+0000 7f888e7c6640 1 --2- 172.21.15.19:0/1263096390 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88700a3bb0 0x7f88700a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8878000c00 tx=0x7f8878031510 comp rx=0 tx=0).stop 2024-04-25T12:05:11.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.537+0000 7f888e7c6640 1 -- 172.21.15.19:0/1263096390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8888102060 msgr2=0x7f88881034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:11.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.537+0000 7f888e7c6640 1 --2- 172.21.15.19:0/1263096390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8888102060 0x7f88881034d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f887c00d770 tx=0x7f887c00dc30 comp rx=0 tx=0).stop 2024-04-25T12:05:11.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.537+0000 7f888e7c6640 1 -- 172.21.15.19:0/1263096390 shutdown_connections 2024-04-25T12:05:11.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.537+0000 7f888e7c6640 1 --2- 172.21.15.19:0/1263096390 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88700a3bb0 0x7f88700a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.537+0000 7f888e7c6640 1 --2- 172.21.15.19:0/1263096390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8888102a20 0x7f8888103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.537+0000 7f888e7c6640 1 --2- 172.21.15.19:0/1263096390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8888102060 0x7f88881034d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.537+0000 7f888e7c6640 1 --2- 172.21.15.19:0/1263096390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8888100f10 0x7f8888104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.538+0000 7f888e7c6640 1 -- 172.21.15.19:0/1263096390 >> 172.21.15.19:0/1263096390 conn(0x7f88880fc4a0 msgr2=0x7f88880fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:11.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.538+0000 7f888e7c6640 1 -- 172.21.15.19:0/1263096390 shutdown_connections 2024-04-25T12:05:11.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.538+0000 7f888e7c6640 1 -- 172.21.15.19:0/1263096390 wait complete. 2024-04-25T12:05:11.546 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:05:11.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.651+0000 7feee2d15640 1 -- 172.21.15.19:0/529537245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feedc100ea0 msgr2=0x7feedc1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.651+0000 7feee2d15640 1 --2- 172.21.15.19:0/529537245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feedc100ea0 0x7feedc1012c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feed0009900 tx=0x7feed002a410 comp rx=0 tx=0).stop 2024-04-25T12:05:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.651+0000 7feee2d15640 1 -- 172.21.15.19:0/529537245 shutdown_connections 2024-04-25T12:05:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.651+0000 7feee2d15640 1 --2- 172.21.15.19:0/529537245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feedc1029b0 0x7feedc1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.651+0000 7feee2d15640 1 --2- 172.21.15.19:0/529537245 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feedc101ff0 0x7feedc102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.651+0000 7feee2d15640 1 --2- 172.21.15.19:0/529537245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feedc100ea0 0x7feedc1012c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.651+0000 7feee2d15640 1 -- 172.21.15.19:0/529537245 >> 172.21.15.19:0/529537245 conn(0x7feedc0fc430 msgr2=0x7feedc0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.651+0000 7feee2d15640 1 -- 172.21.15.19:0/529537245 shutdown_connections 2024-04-25T12:05:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.651+0000 7feee2d15640 1 -- 172.21.15.19:0/529537245 wait complete. 2024-04-25T12:05:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.651+0000 7feee2d15640 1 Processor -- start 2024-04-25T12:05:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.651+0000 7feee2d15640 1 -- start start 2024-04-25T12:05:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee2d15640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feedc100ea0 0x7feedc1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:11.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee2d15640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feedc101ff0 0x7feedc1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee2d15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feedc1029b0 0x7feedc19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee2d15640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feedc10cb10 con 0x7feedc1029b0 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee2d15640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feedc10c990 con 0x7feedc100ea0 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee2d15640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7feedc10cc90 con 0x7feedc101ff0 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee12b2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feedc1029b0 0x7feedc19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee12b2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feedc1029b0 0x7feedc19f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42646/0 (socket says 172.21.15.19:42646) 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee12b2640 1 -- 172.21.15.19:0/2290166817 learned_addr learned my addr 172.21.15.19:0/2290166817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feedbfff640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feedc101ff0 0x7feedc1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee0ab1640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feedc100ea0 0x7feedc1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee12b2640 1 -- 172.21.15.19:0/2290166817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feedc101ff0 msgr2=0x7feedc1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee12b2640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feedc101ff0 0x7feedc1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee12b2640 1 -- 172.21.15.19:0/2290166817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feedc100ea0 msgr2=0x7feedc1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee12b2640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feedc100ea0 0x7feedc1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee12b2640 1 -- 172.21.15.19:0/2290166817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feedc19fed0 con 0x7feedc1029b0 2024-04-25T12:05:11.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feedbfff640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feedc101ff0 0x7feedc1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:05:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee12b2640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feedc1029b0 0x7feedc19f860 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7feed400eaa0 tx=0x7feed400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feed9ffb640 1 -- 172.21.15.19:0/2290166817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed400cdb0 con 0x7feedc1029b0 2024-04-25T12:05:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feee2d15640 1 -- 172.21.15.19:0/2290166817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feedc1a01c0 con 0x7feedc1029b0 2024-04-25T12:05:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.652+0000 7feed9ffb640 1 -- 172.21.15.19:0/2290166817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7feed4005020 con 0x7feedc1029b0 2024-04-25T12:05:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.653+0000 7feed9ffb640 1 -- 172.21.15.19:0/2290166817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40116d0 con 0x7feedc1029b0 2024-04-25T12:05:11.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.653+0000 7feee2d15640 1 -- 172.21.15.19:0/2290166817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feedc1a06c0 con 0x7feedc1029b0 2024-04-25T12:05:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.654+0000 7feee2d15640 1 -- 172.21.15.19:0/2290166817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feedc1a0980 con 0x7feedc1029b0 2024-04-25T12:05:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.655+0000 7feed9ffb640 1 -- 172.21.15.19:0/2290166817 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feed40051d0 con 0x7feedc1029b0 2024-04-25T12:05:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.655+0000 7feed9ffb640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7feecc0a3bb0 0x7feecc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.655+0000 7feee0ab1640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7feecc0a3bb0 0x7feecc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.655+0000 7feed9ffb640 1 -- 172.21.15.19:0/2290166817 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7feed40d0440 con 0x7feedc1029b0 2024-04-25T12:05:11.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.656+0000 7feee0ab1640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7feecc0a3bb0 0x7feecc0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feed0000c00 tx=0x7feed0031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:11.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.660+0000 7feed9ffb640 1 -- 172.21.15.19:0/2290166817 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feed4016030 con 0x7feedc1029b0 2024-04-25T12:05:11.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.724+0000 7feed9ffb640 1 -- 172.21.15.19:0/2290166817 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feed4098f90 con 0x7feedc1029b0 2024-04-25T12:05:11.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:11 smithi019 ceph-mon[29213]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:11.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:11 smithi019 ceph-mon[29213]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:11.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:11 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] up:active 2024-04-25T12:05:11.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:11 smithi019 ceph-mon[29213]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T12:05:11.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:11 smithi019 ceph-mon[29213]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T12:05:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.897+0000 7feee2d15640 1 -- 172.21.15.19:0/2290166817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feedc1a0b50 con 0x7feedc1029b0 2024-04-25T12:05:11.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.898+0000 7feed9ffb640 1 -- 172.21.15.19:0/2290166817 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11944 (secure 0 0 0) 0x7feed40a3410 con 0x7feedc1029b0 2024-04-25T12:05:11.896 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:05:11.896 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T12:05:04.008679+0000","modified":"2024-04-25T12:05:11.186222+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24514,"mds_1":24500,"mds_2":24496,"mds_3":24484,"mds_4":14655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/1694897630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1694897630},{"type":"v1","addr":"172.21.15.19:6839","nonce":1694897630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6841/3247157172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3247157172},{"type":"v1","addr":"172.21.15.19:6841","nonce":3247157172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/3918801450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3918801450},{"type":"v1","addr":"172.21.15.150:6837","nonce":3918801450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2553528303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2553528303},{"type":"v1","addr":"172.21.15.150:6839","nonce":2553528303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/1723439500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1723439500},{"type":"v1","addr":"172.21.15.169:6839","nonce":1723439500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24500,24514,24496,24484,14655]},"id":1}]} 2024-04-25T12:05:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-25T12:05:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.900+0000 7feee2d15640 1 -- 172.21.15.19:0/2290166817 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7feecc0a3bb0 msgr2=0x7feecc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.900+0000 7feee2d15640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7feecc0a3bb0 0x7feecc0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feed0000c00 tx=0x7feed0031510 comp rx=0 tx=0).stop 2024-04-25T12:05:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.900+0000 7feee2d15640 1 -- 172.21.15.19:0/2290166817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feedc1029b0 msgr2=0x7feedc19f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:11.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.900+0000 7feee2d15640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feedc1029b0 0x7feedc19f860 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7feed400eaa0 tx=0x7feed400ef60 comp rx=0 tx=0).stop 2024-04-25T12:05:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.901+0000 7feee2d15640 1 -- 172.21.15.19:0/2290166817 shutdown_connections 2024-04-25T12:05:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.901+0000 7feee2d15640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7feecc0a3bb0 0x7feecc0a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.901+0000 7feee2d15640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feedc1029b0 0x7feedc19f860 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.901+0000 7feee2d15640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feedc101ff0 0x7feedc1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.901+0000 7feee2d15640 1 --2- 172.21.15.19:0/2290166817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feedc100ea0 0x7feedc1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.901+0000 7feee2d15640 1 -- 172.21.15.19:0/2290166817 >> 172.21.15.19:0/2290166817 conn(0x7feedc0fc430 msgr2=0x7feedc0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.901+0000 7feee2d15640 1 -- 172.21.15.19:0/2290166817 shutdown_connections 2024-04-25T12:05:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:11.901+0000 7feee2d15640 1 -- 172.21.15.19:0/2290166817 wait complete. 2024-04-25T12:05:11.909 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:05:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.013+0000 7f4b9035d640 1 -- 172.21.15.19:0/3817590138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88107c70 msgr2=0x7f4b8810a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.013+0000 7f4b9035d640 1 --2- 172.21.15.19:0/3817590138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88107c70 0x7f4b8810a0c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009900 tx=0x7f4b8002a410 comp rx=0 tx=0).stop 2024-04-25T12:05:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.013+0000 7f4b9035d640 1 -- 172.21.15.19:0/3817590138 shutdown_connections 2024-04-25T12:05:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.013+0000 7f4b9035d640 1 --2- 172.21.15.19:0/3817590138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88107c70 0x7f4b8810a0c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.013+0000 7f4b9035d640 1 --2- 172.21.15.19:0/3817590138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b88105340 0x7f4b88107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.013+0000 7f4b9035d640 1 --2- 172.21.15.19:0/3817590138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b88102a10 0x7f4b88104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.013+0000 7f4b9035d640 1 -- 172.21.15.19:0/3817590138 >> 172.21.15.19:0/3817590138 conn(0x7f4b880fc400 msgr2=0x7f4b880fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.013+0000 7f4b9035d640 1 -- 172.21.15.19:0/3817590138 shutdown_connections 2024-04-25T12:05:12.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.013+0000 7f4b9035d640 1 -- 172.21.15.19:0/3817590138 wait complete. 2024-04-25T12:05:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b9035d640 1 Processor -- start 2024-04-25T12:05:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b9035d640 1 -- start start 2024-04-25T12:05:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b9035d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88102a10 0x7f4b88198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b9035d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b88105340 0x7f4b88198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b9035d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b88107c70 0x7f4b8819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b9035d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810dc50 con 0x7f4b88102a10 2024-04-25T12:05:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b9035d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810dad0 con 0x7f4b88105340 2024-04-25T12:05:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b9035d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810ddd0 con 0x7f4b88107c70 2024-04-25T12:05:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b8e0f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88102a10 0x7f4b88198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b8e0f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88102a10 0x7f4b88198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42676/0 (socket says 172.21.15.19:42676) 2024-04-25T12:05:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b8e0f9640 1 -- 172.21.15.19:0/1451737908 learned_addr learned my addr 172.21.15.19:0/1451737908 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:12.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b8e8fa640 1 --2- 172.21.15.19:0/1451737908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b88107c70 0x7f4b8819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b8d8f8640 1 --2- 172.21.15.19:0/1451737908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b88105340 0x7f4b88198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b8e0f9640 1 -- 172.21.15.19:0/1451737908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b88107c70 msgr2=0x7f4b8819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b8e0f9640 1 --2- 172.21.15.19:0/1451737908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b88107c70 0x7f4b8819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b8e0f9640 1 -- 172.21.15.19:0/1451737908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b88105340 msgr2=0x7f4b88198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b8e0f9640 1 --2- 172.21.15.19:0/1451737908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b88105340 0x7f4b88198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.014+0000 7f4b8e0f9640 1 -- 172.21.15.19:0/1451737908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8819fe60 con 0x7f4b88102a10 2024-04-25T12:05:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.015+0000 7f4b8e0f9640 1 --2- 172.21.15.19:0/1451737908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88102a10 0x7f4b88198240 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800cce0 tx=0x7f4b7800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.015+0000 7f4b877fe640 1 -- 172.21.15.19:0/1451737908 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78004b20 con 0x7f4b88102a10 2024-04-25T12:05:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.015+0000 7f4b877fe640 1 -- 172.21.15.19:0/1451737908 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4b78004cc0 con 0x7f4b88102a10 2024-04-25T12:05:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.015+0000 7f4b9035d640 1 -- 172.21.15.19:0/1451737908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b881a0150 con 0x7f4b88102a10 2024-04-25T12:05:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.015+0000 7f4b877fe640 1 -- 172.21.15.19:0/1451737908 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7800f760 con 0x7f4b88102a10 2024-04-25T12:05:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.015+0000 7f4b9035d640 1 -- 172.21.15.19:0/1451737908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881a06a0 con 0x7f4b88102a10 2024-04-25T12:05:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.016+0000 7f4b9035d640 1 -- 172.21.15.19:0/1451737908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b880fdef0 con 0x7f4b88102a10 2024-04-25T12:05:12.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.017+0000 7f4b877fe640 1 -- 172.21.15.19:0/1451737908 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4b78005580 con 0x7f4b88102a10 2024-04-25T12:05:12.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.018+0000 7f4b877fe640 1 --2- 172.21.15.19:0/1451737908 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4b700a3c40 0x7f4b700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:12.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.018+0000 7f4b8d8f8640 1 --2- 172.21.15.19:0/1451737908 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4b700a3c40 0x7f4b700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:12.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.018+0000 7f4b877fe640 1 -- 172.21.15.19:0/1451737908 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f4b780d0530 con 0x7f4b88102a10 2024-04-25T12:05:12.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.018+0000 7f4b8d8f8640 1 --2- 172.21.15.19:0/1451737908 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4b700a3c40 0x7f4b700a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4b88199750 tx=0x7f4b7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:12.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.022+0000 7f4b877fe640 1 -- 172.21.15.19:0/1451737908 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b78014030 con 0x7f4b88102a10 2024-04-25T12:05:12.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:11 smithi169 ceph-mon[32710]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:12.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:11 smithi169 ceph-mon[32710]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:12.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:11 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] up:active 2024-04-25T12:05:12.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:11 smithi169 ceph-mon[32710]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T12:05:12.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:11 smithi169 ceph-mon[32710]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T12:05:12.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:11 smithi150 ceph-mon[32250]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:12.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:11 smithi150 ceph-mon[32250]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:12.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:11 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] up:active 2024-04-25T12:05:12.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:11 smithi150 ceph-mon[32250]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T12:05:12.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:11 smithi150 ceph-mon[32250]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T12:05:12.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.209+0000 7f4b9035d640 1 -- 172.21.15.19:0/1451737908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b88062b50 con 0x7f4b88102a10 2024-04-25T12:05:12.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.210+0000 7f4b877fe640 1 -- 172.21.15.19:0/1451737908 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22203 (secure 0 0 0) 0x7f4b7809ce60 con 0x7f4b88102a10 2024-04-25T12:05:12.208 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:05:12.208 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:05:05.642027+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:05:01.658354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:05:02.969078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:05:12.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.212+0000 7f4b9035d640 1 -- 172.21.15.19:0/1451737908 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4b700a3c40 msgr2=0x7f4b700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.212+0000 7f4b9035d640 1 --2- 172.21.15.19:0/1451737908 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4b700a3c40 0x7f4b700a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4b88199750 tx=0x7f4b7c005f90 comp rx=0 tx=0).stop 2024-04-25T12:05:12.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.212+0000 7f4b9035d640 1 -- 172.21.15.19:0/1451737908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88102a10 msgr2=0x7f4b88198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.212+0000 7f4b9035d640 1 --2- 172.21.15.19:0/1451737908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88102a10 0x7f4b88198240 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800cce0 tx=0x7f4b7800a620 comp rx=0 tx=0).stop 2024-04-25T12:05:12.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.212+0000 7f4b9035d640 1 -- 172.21.15.19:0/1451737908 shutdown_connections 2024-04-25T12:05:12.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.212+0000 7f4b9035d640 1 --2- 172.21.15.19:0/1451737908 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4b700a3c40 0x7f4b700a6100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.212+0000 7f4b9035d640 1 --2- 172.21.15.19:0/1451737908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b88107c70 0x7f4b8819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.212+0000 7f4b9035d640 1 --2- 172.21.15.19:0/1451737908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b88105340 0x7f4b88198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.212+0000 7f4b9035d640 1 --2- 172.21.15.19:0/1451737908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88102a10 0x7f4b88198240 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.212+0000 7f4b9035d640 1 -- 172.21.15.19:0/1451737908 >> 172.21.15.19:0/1451737908 conn(0x7f4b880fc400 msgr2=0x7f4b88103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.213+0000 7f4b9035d640 1 -- 172.21.15.19:0/1451737908 shutdown_connections 2024-04-25T12:05:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.213+0000 7f4b9035d640 1 -- 172.21.15.19:0/1451737908 wait complete. 2024-04-25T12:05:12.222 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T12:05:12.222 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:05:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.325+0000 7fe3220d0640 1 -- 172.21.15.19:0/1927200655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe31c103b70 msgr2=0x7fe31c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.325+0000 7fe3220d0640 1 --2- 172.21.15.19:0/1927200655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe31c103b70 0x7fe31c10a400 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe314009ca0 tx=0x7fe31402a8c0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 -- 172.21.15.19:0/1927200655 shutdown_connections 2024-04-25T12:05:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 --2- 172.21.15.19:0/1927200655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe31c103b70 0x7fe31c10a400 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 --2- 172.21.15.19:0/1927200655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31c1031b0 0x7fe31c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 --2- 172.21.15.19:0/1927200655 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe31c101fc0 0x7fe31c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 -- 172.21.15.19:0/1927200655 >> 172.21.15.19:0/1927200655 conn(0x7fe31c0fd550 msgr2=0x7fe31c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 -- 172.21.15.19:0/1927200655 shutdown_connections 2024-04-25T12:05:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 -- 172.21.15.19:0/1927200655 wait complete. 2024-04-25T12:05:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 Processor -- start 2024-04-25T12:05:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 -- start start 2024-04-25T12:05:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe31c101fc0 0x7fe31c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31c1031b0 0x7fe31c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe31c103b70 0x7fe31c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c10dc20 con 0x7fe31c101fc0 2024-04-25T12:05:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c10daa0 con 0x7fe31c1031b0 2024-04-25T12:05:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.326+0000 7fe3220d0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c10dda0 con 0x7fe31c103b70 2024-04-25T12:05:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe31b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe31c101fc0 0x7fe31c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe31b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe31c101fc0 0x7fe31c198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42696/0 (socket says 172.21.15.19:42696) 2024-04-25T12:05:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe31b7fe640 1 -- 172.21.15.19:0/3576184128 learned_addr learned my addr 172.21.15.19:0/3576184128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe31affd640 1 --2- 172.21.15.19:0/3576184128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31c1031b0 0x7fe31c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe31bfff640 1 --2- 172.21.15.19:0/3576184128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe31c103b70 0x7fe31c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe31b7fe640 1 -- 172.21.15.19:0/3576184128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe31c103b70 msgr2=0x7fe31c19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe31b7fe640 1 --2- 172.21.15.19:0/3576184128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe31c103b70 0x7fe31c19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe31b7fe640 1 -- 172.21.15.19:0/3576184128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31c1031b0 msgr2=0x7fe31c198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe31b7fe640 1 --2- 172.21.15.19:0/3576184128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31c1031b0 0x7fe31c198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe31b7fe640 1 -- 172.21.15.19:0/3576184128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31c19fe60 con 0x7fe31c101fc0 2024-04-25T12:05:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe31b7fe640 1 --2- 172.21.15.19:0/3576184128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe31c101fc0 0x7fe31c198240 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00c970 tx=0x7fe30c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe318ff9640 1 -- 172.21.15.19:0/3576184128 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c004220 con 0x7fe31c101fc0 2024-04-25T12:05:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe318ff9640 1 -- 172.21.15.19:0/3576184128 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe30c0043c0 con 0x7fe31c101fc0 2024-04-25T12:05:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe3220d0640 1 -- 172.21.15.19:0/3576184128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe31c1a0150 con 0x7fe31c101fc0 2024-04-25T12:05:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe318ff9640 1 -- 172.21.15.19:0/3576184128 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c004a00 con 0x7fe31c101fc0 2024-04-25T12:05:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.327+0000 7fe3220d0640 1 -- 172.21.15.19:0/3576184128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe31c1a0650 con 0x7fe31c101fc0 2024-04-25T12:05:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.328+0000 7fe3220d0640 1 -- 172.21.15.19:0/3576184128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe31c109280 con 0x7fe31c101fc0 2024-04-25T12:05:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.330+0000 7fe318ff9640 1 -- 172.21.15.19:0/3576184128 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe30c0079b0 con 0x7fe31c101fc0 2024-04-25T12:05:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.330+0000 7fe318ff9640 1 --2- 172.21.15.19:0/3576184128 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe3040a3c40 0x7fe3040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.330+0000 7fe31affd640 1 --2- 172.21.15.19:0/3576184128 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe3040a3c40 0x7fe3040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.330+0000 7fe318ff9640 1 -- 172.21.15.19:0/3576184128 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fe30c0d0680 con 0x7fe31c101fc0 2024-04-25T12:05:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.331+0000 7fe31affd640 1 --2- 172.21.15.19:0/3576184128 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe3040a3c40 0x7fe3040a6100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe31c199750 tx=0x7fe31000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.334+0000 7fe318ff9640 1 -- 172.21.15.19:0/3576184128 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe30c014030 con 0x7fe31c101fc0 2024-04-25T12:05:12.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.516+0000 7fe3220d0640 1 -- 172.21.15.19:0/3576184128 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe31c062390 con 0x7fe3040a3c40 2024-04-25T12:05:12.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.524+0000 7fe318ff9640 1 -- 172.21.15.19:0/3576184128 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258898 (secure 0 0 0) 0x7fe31c062390 con 0x7fe3040a3c40 2024-04-25T12:05:12.525 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:05:12.527 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T12:05:12.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.527+0000 7fe3220d0640 1 -- 172.21.15.19:0/3576184128 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe3040a3c40 msgr2=0x7fe3040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.527+0000 7fe3220d0640 1 --2- 172.21.15.19:0/3576184128 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe3040a3c40 0x7fe3040a6100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe31c199750 tx=0x7fe31000a4c0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.527+0000 7fe3220d0640 1 -- 172.21.15.19:0/3576184128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe31c101fc0 msgr2=0x7fe31c198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.527+0000 7fe3220d0640 1 --2- 172.21.15.19:0/3576184128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe31c101fc0 0x7fe31c198240 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00c970 tx=0x7fe30c00ce30 comp rx=0 tx=0).stop 2024-04-25T12:05:12.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.527+0000 7fe3220d0640 1 -- 172.21.15.19:0/3576184128 shutdown_connections 2024-04-25T12:05:12.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.527+0000 7fe3220d0640 1 --2- 172.21.15.19:0/3576184128 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe3040a3c40 0x7fe3040a6100 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.527+0000 7fe3220d0640 1 --2- 172.21.15.19:0/3576184128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe31c103b70 0x7fe31c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.527+0000 7fe3220d0640 1 --2- 172.21.15.19:0/3576184128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe31c1031b0 0x7fe31c198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.527+0000 7fe3220d0640 1 --2- 172.21.15.19:0/3576184128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe31c101fc0 0x7fe31c198240 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.527+0000 7fe3220d0640 1 -- 172.21.15.19:0/3576184128 >> 172.21.15.19:0/3576184128 conn(0x7fe31c0fd550 msgr2=0x7fe31c106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:12.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.527+0000 7fe3220d0640 1 -- 172.21.15.19:0/3576184128 shutdown_connections 2024-04-25T12:05:12.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.527+0000 7fe3220d0640 1 -- 172.21.15.19:0/3576184128 wait complete. 2024-04-25T12:05:12.535 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-04-25T12:05:10.896634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595320,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":141,"num_read_kb":123,"num_write":279,"num_write_kb":2967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":239,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364380,"kb_used_data":11028,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577444,"statfs":{"total":1151940427776,"available":1151567302656,"internally_reserved":0,"allocated":11292672,"data_stored":7057061,"data_compressed":107301,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4952,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":98,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.255929"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.655237+0000","last_change":"2024-04-25T12:05:04.662228+0000","last_active":"2024-04-25T12:05:05.655237+0000","last_peered":"2024-04-25T12:05:05.655237+0000","last_clean":"2024-04-25T12:05:05.655237+0000","last_became_active":"2024-04-25T12:05:04.661855+0000","last_became_peered":"2024-04-25T12:05:04.661855+0000","last_unstale":"2024-04-25T12:05:05.655237+0000","last_undegraded":"2024-04-25T12:05:05.655237+0000","last_fullsized":"2024-04-25T12:05:05.655237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:41.662330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756444+0000","last_change":"2024-04-25T12:05:04.664103+0000","last_active":"2024-04-25T12:05:06.756444+0000","last_peered":"2024-04-25T12:05:06.756444+0000","last_clean":"2024-04-25T12:05:06.756444+0000","last_became_active":"2024-04-25T12:05:04.661946+0000","last_became_peered":"2024-04-25T12:05:04.661946+0000","last_unstale":"2024-04-25T12:05:06.756444+0000","last_undegraded":"2024-04-25T12:05:06.756444+0000","last_fullsized":"2024-04-25T12:05:06.756444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:08.119278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756352+0000","last_change":"2024-04-25T12:05:04.667608+0000","last_active":"2024-04-25T12:05:06.756352+0000","last_peered":"2024-04-25T12:05:06.756352+0000","last_clean":"2024-04-25T12:05:06.756352+0000","last_became_active":"2024-04-25T12:05:04.667227+0000","last_became_peered":"2024-04-25T12:05:04.667227+0000","last_unstale":"2024-04-25T12:05:06.756352+0000","last_undegraded":"2024-04-25T12:05:06.756352+0000","last_fullsized":"2024-04-25T12:05:06.756352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:17.794368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756489+0000","last_change":"2024-04-25T12:05:04.671431+0000","last_active":"2024-04-25T12:05:06.756489+0000","last_peered":"2024-04-25T12:05:06.756489+0000","last_clean":"2024-04-25T12:05:06.756489+0000","last_became_active":"2024-04-25T12:05:04.670876+0000","last_became_peered":"2024-04-25T12:05:04.670876+0000","last_unstale":"2024-04-25T12:05:06.756489+0000","last_undegraded":"2024-04-25T12:05:06.756489+0000","last_fullsized":"2024-04-25T12:05:06.756489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:03.564092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683251+0000","last_change":"2024-04-25T12:05:04.665061+0000","last_active":"2024-04-25T12:05:05.683251+0000","last_peered":"2024-04-25T12:05:05.683251+0000","last_clean":"2024-04-25T12:05:05.683251+0000","last_became_active":"2024-04-25T12:05:04.664705+0000","last_became_peered":"2024-04-25T12:05:04.664705+0000","last_unstale":"2024-04-25T12:05:05.683251+0000","last_undegraded":"2024-04-25T12:05:05.683251+0000","last_fullsized":"2024-04-25T12:05:05.683251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:16.434402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.648451+0000","last_change":"2024-04-25T12:05:04.668285+0000","last_active":"2024-04-25T12:05:05.648451+0000","last_peered":"2024-04-25T12:05:05.648451+0000","last_clean":"2024-04-25T12:05:05.648451+0000","last_became_active":"2024-04-25T12:05:04.667941+0000","last_became_peered":"2024-04-25T12:05:04.667941+0000","last_unstale":"2024-04-25T12:05:05.648451+0000","last_undegraded":"2024-04-25T12:05:05.648451+0000","last_fullsized":"2024-04-25T12:05:05.648451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:02.527280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014139+0000","last_change":"2024-04-25T12:05:04.661283+0000","last_active":"2024-04-25T12:05:06.014139+0000","last_peered":"2024-04-25T12:05:06.014139+0000","last_clean":"2024-04-25T12:05:06.014139+0000","last_became_active":"2024-04-25T12:05:04.660944+0000","last_became_peered":"2024-04-25T12:05:04.660944+0000","last_unstale":"2024-04-25T12:05:06.014139+0000","last_undegraded":"2024-04-25T12:05:06.014139+0000","last_fullsized":"2024-04-25T12:05:06.014139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:37.286135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683124+0000","last_change":"2024-04-25T12:05:04.672998+0000","last_active":"2024-04-25T12:05:05.683124+0000","last_peered":"2024-04-25T12:05:05.683124+0000","last_clean":"2024-04-25T12:05:05.683124+0000","last_became_active":"2024-04-25T12:05:04.672611+0000","last_became_peered":"2024-04-25T12:05:04.672611+0000","last_unstale":"2024-04-25T12:05:05.683124+0000","last_undegraded":"2024-04-25T12:05:05.683124+0000","last_fullsized":"2024-04-25T12:05:05.683124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:22.926657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.636465+0000","last_change":"2024-04-25T12:05:04.662957+0000","last_active":"2024-04-25T12:05:06.636465+0000","last_peered":"2024-04-25T12:05:06.636465+0000","last_clean":"2024-04-25T12:05:06.636465+0000","last_became_active":"2024-04-25T12:05:04.662636+0000","last_became_peered":"2024-04-25T12:05:04.662636+0000","last_unstale":"2024-04-25T12:05:06.636465+0000","last_undegraded":"2024-04-25T12:05:06.636465+0000","last_fullsized":"2024-04-25T12:05:06.636465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:05.209057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.648285+0000","last_change":"2024-04-25T12:05:04.670076+0000","last_active":"2024-04-25T12:05:05.648285+0000","last_peered":"2024-04-25T12:05:05.648285+0000","last_clean":"2024-04-25T12:05:05.648285+0000","last_became_active":"2024-04-25T12:05:04.669659+0000","last_became_peered":"2024-04-25T12:05:04.669659+0000","last_unstale":"2024-04-25T12:05:05.648285+0000","last_undegraded":"2024-04-25T12:05:05.648285+0000","last_fullsized":"2024-04-25T12:05:05.648285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:06.503269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014504+0000","last_change":"2024-04-25T12:05:04.664849+0000","last_active":"2024-04-25T12:05:06.014504+0000","last_peered":"2024-04-25T12:05:06.014504+0000","last_clean":"2024-04-25T12:05:06.014504+0000","last_became_active":"2024-04-25T12:05:04.664322+0000","last_became_peered":"2024-04-25T12:05:04.664322+0000","last_unstale":"2024-04-25T12:05:06.014504+0000","last_undegraded":"2024-04-25T12:05:06.014504+0000","last_fullsized":"2024-04-25T12:05:06.014504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:59:03.872076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756448+0000","last_change":"2024-04-25T12:05:04.669403+0000","last_active":"2024-04-25T12:05:06.756448+0000","last_peered":"2024-04-25T12:05:06.756448+0000","last_clean":"2024-04-25T12:05:06.756448+0000","last_became_active":"2024-04-25T12:05:04.669052+0000","last_became_peered":"2024-04-25T12:05:04.669052+0000","last_unstale":"2024-04-25T12:05:06.756448+0000","last_undegraded":"2024-04-25T12:05:06.756448+0000","last_fullsized":"2024-04-25T12:05:06.756448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:18.997019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.647243+0000","last_change":"2024-04-25T12:05:04.664312+0000","last_active":"2024-04-25T12:05:05.647243+0000","last_peered":"2024-04-25T12:05:05.647243+0000","last_clean":"2024-04-25T12:05:05.647243+0000","last_became_active":"2024-04-25T12:05:04.663980+0000","last_became_peered":"2024-04-25T12:05:04.663980+0000","last_unstale":"2024-04-25T12:05:05.647243+0000","last_undegraded":"2024-04-25T12:05:05.647243+0000","last_fullsized":"2024-04-25T12:05:05.647243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:58.427231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756149+0000","last_change":"2024-04-25T12:05:04.663328+0000","last_active":"2024-04-25T12:05:06.756149+0000","last_peered":"2024-04-25T12:05:06.756149+0000","last_clean":"2024-04-25T12:05:06.756149+0000","last_became_active":"2024-04-25T12:05:04.662575+0000","last_became_peered":"2024-04-25T12:05:04.662575+0000","last_unstale":"2024-04-25T12:05:06.756149+0000","last_undegraded":"2024-04-25T12:05:06.756149+0000","last_fullsized":"2024-04-25T12:05:06.756149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:41.786213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756095+0000","last_change":"2024-04-25T12:05:04.672332+0000","last_active":"2024-04-25T12:05:06.756095+0000","last_peered":"2024-04-25T12:05:06.756095+0000","last_clean":"2024-04-25T12:05:06.756095+0000","last_became_active":"2024-04-25T12:05:04.671951+0000","last_became_peered":"2024-04-25T12:05:04.671951+0000","last_unstale":"2024-04-25T12:05:06.756095+0000","last_undegraded":"2024-04-25T12:05:06.756095+0000","last_fullsized":"2024-04-25T12:05:06.756095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:27.283686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.635952+0000","last_change":"2024-04-25T12:05:04.669224+0000","last_active":"2024-04-25T12:05:06.635952+0000","last_peered":"2024-04-25T12:05:06.635952+0000","last_clean":"2024-04-25T12:05:06.635952+0000","last_became_active":"2024-04-25T12:05:04.668894+0000","last_became_peered":"2024-04-25T12:05:04.668894+0000","last_unstale":"2024-04-25T12:05:06.635952+0000","last_undegraded":"2024-04-25T12:05:06.635952+0000","last_fullsized":"2024-04-25T12:05:06.635952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:19.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.655361+0000","last_change":"2024-04-25T12:05:04.670391+0000","last_active":"2024-04-25T12:05:05.655361+0000","last_peered":"2024-04-25T12:05:05.655361+0000","last_clean":"2024-04-25T12:05:05.655361+0000","last_became_active":"2024-04-25T12:05:04.670021+0000","last_became_peered":"2024-04-25T12:05:04.670021+0000","last_unstale":"2024-04-25T12:05:05.655361+0000","last_undegraded":"2024-04-25T12:05:05.655361+0000","last_fullsized":"2024-04-25T12:05:05.655361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:40.214491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.681536+0000","last_change":"2024-04-25T12:05:04.665102+0000","last_active":"2024-04-25T12:05:05.681536+0000","last_peered":"2024-04-25T12:05:05.681536+0000","last_clean":"2024-04-25T12:05:05.681536+0000","last_became_active":"2024-04-25T12:05:04.664719+0000","last_became_peered":"2024-04-25T12:05:04.664719+0000","last_unstale":"2024-04-25T12:05:05.681536+0000","last_undegraded":"2024-04-25T12:05:05.681536+0000","last_fullsized":"2024-04-25T12:05:05.681536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:37.528031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.650833+0000","last_change":"2024-04-25T12:05:04.667931+0000","last_active":"2024-04-25T12:05:05.650833+0000","last_peered":"2024-04-25T12:05:05.650833+0000","last_clean":"2024-04-25T12:05:05.650833+0000","last_became_active":"2024-04-25T12:05:04.667606+0000","last_became_peered":"2024-04-25T12:05:04.667606+0000","last_unstale":"2024-04-25T12:05:05.650833+0000","last_undegraded":"2024-04-25T12:05:05.650833+0000","last_fullsized":"2024-04-25T12:05:05.650833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:43.265302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.681476+0000","last_change":"2024-04-25T12:05:03.656983+0000","last_active":"2024-04-25T12:05:05.681476+0000","last_peered":"2024-04-25T12:05:05.681476+0000","last_clean":"2024-04-25T12:05:05.681476+0000","last_became_active":"2024-04-25T12:05:03.656596+0000","last_became_peered":"2024-04-25T12:05:03.656596+0000","last_unstale":"2024-04-25T12:05:05.681476+0000","last_undegraded":"2024-04-25T12:05:05.681476+0000","last_fullsized":"2024-04-25T12:05:05.681476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:25.133153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683650+0000","last_change":"2024-04-25T12:05:04.668880+0000","last_active":"2024-04-25T12:05:05.683650+0000","last_peered":"2024-04-25T12:05:05.683650+0000","last_clean":"2024-04-25T12:05:05.683650+0000","last_became_active":"2024-04-25T12:05:04.668463+0000","last_became_peered":"2024-04-25T12:05:04.668463+0000","last_unstale":"2024-04-25T12:05:05.683650+0000","last_undegraded":"2024-04-25T12:05:05.683650+0000","last_fullsized":"2024-04-25T12:05:05.683650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:52.647241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683697+0000","last_change":"2024-04-25T12:05:03.663928+0000","last_active":"2024-04-25T12:05:05.683697+0000","last_peered":"2024-04-25T12:05:05.683697+0000","last_clean":"2024-04-25T12:05:05.683697+0000","last_became_active":"2024-04-25T12:05:03.663611+0000","last_became_peered":"2024-04-25T12:05:03.663611+0000","last_unstale":"2024-04-25T12:05:05.683697+0000","last_undegraded":"2024-04-25T12:05:05.683697+0000","last_fullsized":"2024-04-25T12:05:05.683697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:26.986245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756710+0000","last_change":"2024-04-25T12:05:04.674426+0000","last_active":"2024-04-25T12:05:06.756710+0000","last_peered":"2024-04-25T12:05:06.756710+0000","last_clean":"2024-04-25T12:05:06.756710+0000","last_became_active":"2024-04-25T12:05:04.668668+0000","last_became_peered":"2024-04-25T12:05:04.668668+0000","last_unstale":"2024-04-25T12:05:06.756710+0000","last_undegraded":"2024-04-25T12:05:06.756710+0000","last_fullsized":"2024-04-25T12:05:06.756710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:34.205038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:09.719974+0000","last_change":"2024-04-25T12:05:03.656464+0000","last_active":"2024-04-25T12:05:09.719974+0000","last_peered":"2024-04-25T12:05:09.719974+0000","last_clean":"2024-04-25T12:05:09.719974+0000","last_became_active":"2024-04-25T12:05:03.656109+0000","last_became_peered":"2024-04-25T12:05:03.656109+0000","last_unstale":"2024-04-25T12:05:09.719974+0000","last_undegraded":"2024-04-25T12:05:09.719974+0000","last_fullsized":"2024-04-25T12:05:09.719974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:32.175997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.636147+0000","last_change":"2024-04-25T12:05:04.662444+0000","last_active":"2024-04-25T12:05:06.636147+0000","last_peered":"2024-04-25T12:05:06.636147+0000","last_clean":"2024-04-25T12:05:06.636147+0000","last_became_active":"2024-04-25T12:05:04.659285+0000","last_became_peered":"2024-04-25T12:05:04.659285+0000","last_unstale":"2024-04-25T12:05:06.636147+0000","last_undegraded":"2024-04-25T12:05:06.636147+0000","last_fullsized":"2024-04-25T12:05:06.636147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:32.678020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683481+0000","last_change":"2024-04-25T12:05:03.656911+0000","last_active":"2024-04-25T12:05:05.683481+0000","last_peered":"2024-04-25T12:05:05.683481+0000","last_clean":"2024-04-25T12:05:05.683481+0000","last_became_active":"2024-04-25T12:05:03.656553+0000","last_became_peered":"2024-04-25T12:05:03.656553+0000","last_unstale":"2024-04-25T12:05:05.683481+0000","last_undegraded":"2024-04-25T12:05:05.683481+0000","last_fullsized":"2024-04-25T12:05:05.683481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:24.154910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756500+0000","last_change":"2024-04-25T12:05:04.674119+0000","last_active":"2024-04-25T12:05:06.756500+0000","last_peered":"2024-04-25T12:05:06.756500+0000","last_clean":"2024-04-25T12:05:06.756500+0000","last_became_active":"2024-04-25T12:05:04.673260+0000","last_became_peered":"2024-04-25T12:05:04.673260+0000","last_unstale":"2024-04-25T12:05:06.756500+0000","last_undegraded":"2024-04-25T12:05:06.756500+0000","last_fullsized":"2024-04-25T12:05:06.756500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:24.941232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:07.703875+0000","last_change":"2024-04-25T12:05:03.658259+0000","last_active":"2024-04-25T12:05:07.703875+0000","last_peered":"2024-04-25T12:05:07.703875+0000","last_clean":"2024-04-25T12:05:07.703875+0000","last_became_active":"2024-04-25T12:05:03.657917+0000","last_became_peered":"2024-04-25T12:05:03.657917+0000","last_unstale":"2024-04-25T12:05:07.703875+0000","last_undegraded":"2024-04-25T12:05:07.703875+0000","last_fullsized":"2024-04-25T12:05:07.703875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:17.780674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683341+0000","last_change":"2024-04-25T12:05:04.657620+0000","last_active":"2024-04-25T12:05:05.683341+0000","last_peered":"2024-04-25T12:05:05.683341+0000","last_clean":"2024-04-25T12:05:05.683341+0000","last_became_active":"2024-04-25T12:05:04.657264+0000","last_became_peered":"2024-04-25T12:05:04.657264+0000","last_unstale":"2024-04-25T12:05:05.683341+0000","last_undegraded":"2024-04-25T12:05:05.683341+0000","last_fullsized":"2024-04-25T12:05:05.683341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:08.159694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.648192+0000","last_change":"2024-04-25T12:05:03.662345+0000","last_active":"2024-04-25T12:05:05.648192+0000","last_peered":"2024-04-25T12:05:05.648192+0000","last_clean":"2024-04-25T12:05:05.648192+0000","last_became_active":"2024-04-25T12:05:03.661843+0000","last_became_peered":"2024-04-25T12:05:03.661843+0000","last_unstale":"2024-04-25T12:05:05.648192+0000","last_undegraded":"2024-04-25T12:05:05.648192+0000","last_fullsized":"2024-04-25T12:05:05.648192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:03.904192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756488+0000","last_change":"2024-04-25T12:05:04.674954+0000","last_active":"2024-04-25T12:05:06.756488+0000","last_peered":"2024-04-25T12:05:06.756488+0000","last_clean":"2024-04-25T12:05:06.756488+0000","last_became_active":"2024-04-25T12:05:04.673652+0000","last_became_peered":"2024-04-25T12:05:04.673652+0000","last_unstale":"2024-04-25T12:05:06.756488+0000","last_undegraded":"2024-04-25T12:05:06.756488+0000","last_fullsized":"2024-04-25T12:05:06.756488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:26.166918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.647011+0000","last_change":"2024-04-25T12:05:03.656638+0000","last_active":"2024-04-25T12:05:05.647011+0000","last_peered":"2024-04-25T12:05:05.647011+0000","last_clean":"2024-04-25T12:05:05.647011+0000","last_became_active":"2024-04-25T12:05:03.656276+0000","last_became_peered":"2024-04-25T12:05:03.656276+0000","last_unstale":"2024-04-25T12:05:05.647011+0000","last_undegraded":"2024-04-25T12:05:05.647011+0000","last_fullsized":"2024-04-25T12:05:05.647011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:30.034633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683036+0000","last_change":"2024-04-25T12:05:04.674146+0000","last_active":"2024-04-25T12:05:05.683036+0000","last_peered":"2024-04-25T12:05:05.683036+0000","last_clean":"2024-04-25T12:05:05.683036+0000","last_became_active":"2024-04-25T12:05:04.673763+0000","last_became_peered":"2024-04-25T12:05:04.673763+0000","last_unstale":"2024-04-25T12:05:05.683036+0000","last_undegraded":"2024-04-25T12:05:05.683036+0000","last_fullsized":"2024-04-25T12:05:05.683036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:09.121546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.755972+0000","last_change":"2024-04-25T12:05:03.658527+0000","last_active":"2024-04-25T12:05:06.755972+0000","last_peered":"2024-04-25T12:05:06.755972+0000","last_clean":"2024-04-25T12:05:06.755972+0000","last_became_active":"2024-04-25T12:05:03.658200+0000","last_became_peered":"2024-04-25T12:05:03.658200+0000","last_unstale":"2024-04-25T12:05:06.755972+0000","last_undegraded":"2024-04-25T12:05:06.755972+0000","last_fullsized":"2024-04-25T12:05:06.755972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:17.367641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.681376+0000","last_change":"2024-04-25T12:05:04.666431+0000","last_active":"2024-04-25T12:05:05.681376+0000","last_peered":"2024-04-25T12:05:05.681376+0000","last_clean":"2024-04-25T12:05:05.681376+0000","last_became_active":"2024-04-25T12:05:04.666012+0000","last_became_peered":"2024-04-25T12:05:04.666012+0000","last_unstale":"2024-04-25T12:05:05.681376+0000","last_undegraded":"2024-04-25T12:05:05.681376+0000","last_fullsized":"2024-04-25T12:05:05.681376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:26.183735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.648144+0000","last_change":"2024-04-25T12:05:03.659730+0000","last_active":"2024-04-25T12:05:05.648144+0000","last_peered":"2024-04-25T12:05:05.648144+0000","last_clean":"2024-04-25T12:05:05.648144+0000","last_became_active":"2024-04-25T12:05:03.659359+0000","last_became_peered":"2024-04-25T12:05:03.659359+0000","last_unstale":"2024-04-25T12:05:05.648144+0000","last_undegraded":"2024-04-25T12:05:05.648144+0000","last_fullsized":"2024-04-25T12:05:05.648144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:16.644771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.636458+0000","last_change":"2024-04-25T12:05:04.666603+0000","last_active":"2024-04-25T12:05:06.636458+0000","last_peered":"2024-04-25T12:05:06.636458+0000","last_clean":"2024-04-25T12:05:06.636458+0000","last_became_active":"2024-04-25T12:05:04.666129+0000","last_became_peered":"2024-04-25T12:05:04.666129+0000","last_unstale":"2024-04-25T12:05:06.636458+0000","last_undegraded":"2024-04-25T12:05:06.636458+0000","last_fullsized":"2024-04-25T12:05:06.636458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:01.024544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:07.703054+0000","last_change":"2024-04-25T12:05:03.660353+0000","last_active":"2024-04-25T12:05:07.703054+0000","last_peered":"2024-04-25T12:05:07.703054+0000","last_clean":"2024-04-25T12:05:07.703054+0000","last_became_active":"2024-04-25T12:05:03.659946+0000","last_became_peered":"2024-04-25T12:05:03.659946+0000","last_unstale":"2024-04-25T12:05:07.703054+0000","last_undegraded":"2024-04-25T12:05:07.703054+0000","last_fullsized":"2024-04-25T12:05:07.703054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:47.492835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756609+0000","last_change":"2024-04-25T12:05:04.669930+0000","last_active":"2024-04-25T12:05:06.756609+0000","last_peered":"2024-04-25T12:05:06.756609+0000","last_clean":"2024-04-25T12:05:06.756609+0000","last_became_active":"2024-04-25T12:05:04.669584+0000","last_became_peered":"2024-04-25T12:05:04.669584+0000","last_unstale":"2024-04-25T12:05:06.756609+0000","last_undegraded":"2024-04-25T12:05:06.756609+0000","last_fullsized":"2024-04-25T12:05:06.756609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:47.250335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756478+0000","last_change":"2024-04-25T12:05:03.661346+0000","last_active":"2024-04-25T12:05:06.756478+0000","last_peered":"2024-04-25T12:05:06.756478+0000","last_clean":"2024-04-25T12:05:06.756478+0000","last_became_active":"2024-04-25T12:05:03.661023+0000","last_became_peered":"2024-04-25T12:05:03.661023+0000","last_unstale":"2024-04-25T12:05:06.756478+0000","last_undegraded":"2024-04-25T12:05:06.756478+0000","last_fullsized":"2024-04-25T12:05:06.756478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:56.659194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.636278+0000","last_change":"2024-04-25T12:05:04.668589+0000","last_active":"2024-04-25T12:05:06.636278+0000","last_peered":"2024-04-25T12:05:06.636278+0000","last_clean":"2024-04-25T12:05:06.636278+0000","last_became_active":"2024-04-25T12:05:04.668124+0000","last_became_peered":"2024-04-25T12:05:04.668124+0000","last_unstale":"2024-04-25T12:05:06.636278+0000","last_undegraded":"2024-04-25T12:05:06.636278+0000","last_fullsized":"2024-04-25T12:05:06.636278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:35.878775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.648326+0000","last_change":"2024-04-25T12:05:03.656287+0000","last_active":"2024-04-25T12:05:05.648326+0000","last_peered":"2024-04-25T12:05:05.648326+0000","last_clean":"2024-04-25T12:05:05.648326+0000","last_became_active":"2024-04-25T12:05:03.655928+0000","last_became_peered":"2024-04-25T12:05:03.655928+0000","last_unstale":"2024-04-25T12:05:05.648326+0000","last_undegraded":"2024-04-25T12:05:05.648326+0000","last_fullsized":"2024-04-25T12:05:05.648326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:53.260268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683564+0000","last_change":"2024-04-25T12:05:04.662866+0000","last_active":"2024-04-25T12:05:05.683564+0000","last_peered":"2024-04-25T12:05:05.683564+0000","last_clean":"2024-04-25T12:05:05.683564+0000","last_became_active":"2024-04-25T12:05:04.661181+0000","last_became_peered":"2024-04-25T12:05:04.661181+0000","last_unstale":"2024-04-25T12:05:05.683564+0000","last_undegraded":"2024-04-25T12:05:05.683564+0000","last_fullsized":"2024-04-25T12:05:05.683564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:46.630686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:10.192473+0000","last_change":"2024-04-25T12:05:03.663558+0000","last_active":"2024-04-25T12:05:10.192473+0000","last_peered":"2024-04-25T12:05:10.192473+0000","last_clean":"2024-04-25T12:05:10.192473+0000","last_became_active":"2024-04-25T12:05:03.661650+0000","last_became_peered":"2024-04-25T12:05:03.661650+0000","last_unstale":"2024-04-25T12:05:10.192473+0000","last_undegraded":"2024-04-25T12:05:10.192473+0000","last_fullsized":"2024-04-25T12:05:10.192473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:27.599992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.647103+0000","last_change":"2024-04-25T12:05:04.661283+0000","last_active":"2024-04-25T12:05:05.647103+0000","last_peered":"2024-04-25T12:05:05.647103+0000","last_clean":"2024-04-25T12:05:05.647103+0000","last_became_active":"2024-04-25T12:05:04.660900+0000","last_became_peered":"2024-04-25T12:05:04.660900+0000","last_unstale":"2024-04-25T12:05:05.647103+0000","last_undegraded":"2024-04-25T12:05:05.647103+0000","last_fullsized":"2024-04-25T12:05:05.647103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:56.670962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:07.703488+0000","last_change":"2024-04-25T12:05:03.662922+0000","last_active":"2024-04-25T12:05:07.703488+0000","last_peered":"2024-04-25T12:05:07.703488+0000","last_clean":"2024-04-25T12:05:07.703488+0000","last_became_active":"2024-04-25T12:05:03.662584+0000","last_became_peered":"2024-04-25T12:05:03.662584+0000","last_unstale":"2024-04-25T12:05:07.703488+0000","last_undegraded":"2024-04-25T12:05:07.703488+0000","last_fullsized":"2024-04-25T12:05:07.703488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:41.664030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014495+0000","last_change":"2024-04-25T12:05:04.672018+0000","last_active":"2024-04-25T12:05:06.014495+0000","last_peered":"2024-04-25T12:05:06.014495+0000","last_clean":"2024-04-25T12:05:06.014495+0000","last_became_active":"2024-04-25T12:05:04.671670+0000","last_became_peered":"2024-04-25T12:05:04.671670+0000","last_unstale":"2024-04-25T12:05:06.014495+0000","last_undegraded":"2024-04-25T12:05:06.014495+0000","last_fullsized":"2024-04-25T12:05:06.014495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:45.005941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756324+0000","last_change":"2024-04-25T12:05:03.656826+0000","last_active":"2024-04-25T12:05:06.756324+0000","last_peered":"2024-04-25T12:05:06.756324+0000","last_clean":"2024-04-25T12:05:06.756324+0000","last_became_active":"2024-04-25T12:05:03.656477+0000","last_became_peered":"2024-04-25T12:05:03.656477+0000","last_unstale":"2024-04-25T12:05:06.756324+0000","last_undegraded":"2024-04-25T12:05:06.756324+0000","last_fullsized":"2024-04-25T12:05:06.756324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:38.111569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014098+0000","last_change":"2024-04-25T12:05:04.671771+0000","last_active":"2024-04-25T12:05:06.014098+0000","last_peered":"2024-04-25T12:05:06.014098+0000","last_clean":"2024-04-25T12:05:06.014098+0000","last_became_active":"2024-04-25T12:05:04.671424+0000","last_became_peered":"2024-04-25T12:05:04.671424+0000","last_unstale":"2024-04-25T12:05:06.014098+0000","last_undegraded":"2024-04-25T12:05:06.014098+0000","last_fullsized":"2024-04-25T12:05:06.014098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:51.327337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'16","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:08.709829+0000","last_change":"2024-04-25T12:05:03.655884+0000","last_active":"2024-04-25T12:05:08.709829+0000","last_peered":"2024-04-25T12:05:08.709829+0000","last_clean":"2024-04-25T12:05:08.709829+0000","last_became_active":"2024-04-25T12:05:03.655437+0000","last_became_peered":"2024-04-25T12:05:03.655437+0000","last_unstale":"2024-04-25T12:05:08.709829+0000","last_undegraded":"2024-04-25T12:05:08.709829+0000","last_fullsized":"2024-04-25T12:05:08.709829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:24.497861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756350+0000","last_change":"2024-04-25T12:05:04.674785+0000","last_active":"2024-04-25T12:05:06.756350+0000","last_peered":"2024-04-25T12:05:06.756350+0000","last_clean":"2024-04-25T12:05:06.756350+0000","last_became_active":"2024-04-25T12:05:04.674469+0000","last_became_peered":"2024-04-25T12:05:04.674469+0000","last_unstale":"2024-04-25T12:05:06.756350+0000","last_undegraded":"2024-04-25T12:05:06.756350+0000","last_fullsized":"2024-04-25T12:05:06.756350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:28.616188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683633+0000","last_change":"2024-04-25T12:05:03.653954+0000","last_active":"2024-04-25T12:05:05.683633+0000","last_peered":"2024-04-25T12:05:05.683633+0000","last_clean":"2024-04-25T12:05:05.683633+0000","last_became_active":"2024-04-25T12:05:03.653583+0000","last_became_peered":"2024-04-25T12:05:03.653583+0000","last_unstale":"2024-04-25T12:05:05.683633+0000","last_undegraded":"2024-04-25T12:05:05.683633+0000","last_fullsized":"2024-04-25T12:05:05.683633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:58.007741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.655154+0000","last_change":"2024-04-25T12:05:04.669395+0000","last_active":"2024-04-25T12:05:05.655154+0000","last_peered":"2024-04-25T12:05:05.655154+0000","last_clean":"2024-04-25T12:05:05.655154+0000","last_became_active":"2024-04-25T12:05:04.669058+0000","last_became_peered":"2024-04-25T12:05:04.669058+0000","last_unstale":"2024-04-25T12:05:05.655154+0000","last_undegraded":"2024-04-25T12:05:05.655154+0000","last_fullsized":"2024-04-25T12:05:05.655154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:03.195889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.681359+0000","last_change":"2024-04-25T12:05:03.658899+0000","last_active":"2024-04-25T12:05:05.681359+0000","last_peered":"2024-04-25T12:05:05.681359+0000","last_clean":"2024-04-25T12:05:05.681359+0000","last_became_active":"2024-04-25T12:05:03.658521+0000","last_became_peered":"2024-04-25T12:05:03.658521+0000","last_unstale":"2024-04-25T12:05:05.681359+0000","last_undegraded":"2024-04-25T12:05:05.681359+0000","last_fullsized":"2024-04-25T12:05:05.681359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:21.296484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.648376+0000","last_change":"2024-04-25T12:05:04.665958+0000","last_active":"2024-04-25T12:05:05.648376+0000","last_peered":"2024-04-25T12:05:05.648376+0000","last_clean":"2024-04-25T12:05:05.648376+0000","last_became_active":"2024-04-25T12:05:04.665619+0000","last_became_peered":"2024-04-25T12:05:04.665619+0000","last_unstale":"2024-04-25T12:05:05.648376+0000","last_undegraded":"2024-04-25T12:05:05.648376+0000","last_fullsized":"2024-04-25T12:05:05.648376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:46.837053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014276+0000","last_change":"2024-04-25T12:05:03.661021+0000","last_active":"2024-04-25T12:05:06.014276+0000","last_peered":"2024-04-25T12:05:06.014276+0000","last_clean":"2024-04-25T12:05:06.014276+0000","last_became_active":"2024-04-25T12:05:03.660690+0000","last_became_peered":"2024-04-25T12:05:03.660690+0000","last_unstale":"2024-04-25T12:05:06.014276+0000","last_undegraded":"2024-04-25T12:05:06.014276+0000","last_fullsized":"2024-04-25T12:05:06.014276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:00.645287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.648224+0000","last_change":"2024-04-25T12:05:04.669496+0000","last_active":"2024-04-25T12:05:05.648224+0000","last_peered":"2024-04-25T12:05:05.648224+0000","last_clean":"2024-04-25T12:05:05.648224+0000","last_became_active":"2024-04-25T12:05:04.669173+0000","last_became_peered":"2024-04-25T12:05:04.669173+0000","last_unstale":"2024-04-25T12:05:05.648224+0000","last_undegraded":"2024-04-25T12:05:05.648224+0000","last_fullsized":"2024-04-25T12:05:05.648224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:30.527192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756547+0000","last_change":"2024-04-25T12:05:04.673622+0000","last_active":"2024-04-25T12:05:06.756547+0000","last_peered":"2024-04-25T12:05:06.756547+0000","last_clean":"2024-04-25T12:05:06.756547+0000","last_became_active":"2024-04-25T12:05:04.673245+0000","last_became_peered":"2024-04-25T12:05:04.673245+0000","last_unstale":"2024-04-25T12:05:06.756547+0000","last_undegraded":"2024-04-25T12:05:06.756547+0000","last_fullsized":"2024-04-25T12:05:06.756547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:59.180387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.646887+0000","last_change":"2024-04-25T12:05:03.665419+0000","last_active":"2024-04-25T12:05:05.646887+0000","last_peered":"2024-04-25T12:05:05.646887+0000","last_clean":"2024-04-25T12:05:05.646887+0000","last_became_active":"2024-04-25T12:05:03.664963+0000","last_became_peered":"2024-04-25T12:05:03.664963+0000","last_unstale":"2024-04-25T12:05:05.646887+0000","last_undegraded":"2024-04-25T12:05:05.646887+0000","last_fullsized":"2024-04-25T12:05:05.646887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:04.074104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683452+0000","last_change":"2024-04-25T12:05:04.674257+0000","last_active":"2024-04-25T12:05:05.683452+0000","last_peered":"2024-04-25T12:05:05.683452+0000","last_clean":"2024-04-25T12:05:05.683452+0000","last_became_active":"2024-04-25T12:05:04.669726+0000","last_became_peered":"2024-04-25T12:05:04.669726+0000","last_unstale":"2024-04-25T12:05:05.683452+0000","last_undegraded":"2024-04-25T12:05:05.683452+0000","last_fullsized":"2024-04-25T12:05:05.683452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:03.304088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756407+0000","last_change":"2024-04-25T12:05:03.652149+0000","last_active":"2024-04-25T12:05:06.756407+0000","last_peered":"2024-04-25T12:05:06.756407+0000","last_clean":"2024-04-25T12:05:06.756407+0000","last_became_active":"2024-04-25T12:05:03.650216+0000","last_became_peered":"2024-04-25T12:05:03.650216+0000","last_unstale":"2024-04-25T12:05:06.756407+0000","last_undegraded":"2024-04-25T12:05:06.756407+0000","last_fullsized":"2024-04-25T12:05:06.756407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:00.255226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014091+0000","last_change":"2024-04-25T12:03:51.170356+0000","last_active":"2024-04-25T12:05:06.014091+0000","last_peered":"2024-04-25T12:05:06.014091+0000","last_clean":"2024-04-25T12:05:06.014091+0000","last_became_active":"2024-04-25T12:03:51.169795+0000","last_became_peered":"2024-04-25T12:03:51.169795+0000","last_unstale":"2024-04-25T12:05:06.014091+0000","last_undegraded":"2024-04-25T12:05:06.014091+0000","last_fullsized":"2024-04-25T12:05:06.014091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:00:21.810858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:00:21.810858+0000","last_clean_scrub_stamp":"2024-04-25T12:00:21.810858+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:34.246289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.646943+0000","last_change":"2024-04-25T12:05:04.671882+0000","last_active":"2024-04-25T12:05:05.646943+0000","last_peered":"2024-04-25T12:05:05.646943+0000","last_clean":"2024-04-25T12:05:05.646943+0000","last_became_active":"2024-04-25T12:05:04.671408+0000","last_became_peered":"2024-04-25T12:05:04.671408+0000","last_unstale":"2024-04-25T12:05:05.646943+0000","last_undegraded":"2024-04-25T12:05:05.646943+0000","last_fullsized":"2024-04-25T12:05:05.646943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:36.465054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:08.708537+0000","last_change":"2024-04-25T12:05:03.659082+0000","last_active":"2024-04-25T12:05:08.708537+0000","last_peered":"2024-04-25T12:05:08.708537+0000","last_clean":"2024-04-25T12:05:08.708537+0000","last_became_active":"2024-04-25T12:05:03.658677+0000","last_became_peered":"2024-04-25T12:05:03.658677+0000","last_unstale":"2024-04-25T12:05:08.708537+0000","last_undegraded":"2024-04-25T12:05:08.708537+0000","last_fullsized":"2024-04-25T12:05:08.708537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:20.938394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:09.721821+0000","last_change":"2024-04-25T12:05:03.661172+0000","last_active":"2024-04-25T12:05:09.721821+0000","last_peered":"2024-04-25T12:05:09.721821+0000","last_clean":"2024-04-25T12:05:09.721821+0000","last_became_active":"2024-04-25T12:05:03.660799+0000","last_became_peered":"2024-04-25T12:05:03.660799+0000","last_unstale":"2024-04-25T12:05:09.721821+0000","last_undegraded":"2024-04-25T12:05:09.721821+0000","last_fullsized":"2024-04-25T12:05:09.721821+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:01.889714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.647180+0000","last_change":"2024-04-25T12:05:04.662203+0000","last_active":"2024-04-25T12:05:05.647180+0000","last_peered":"2024-04-25T12:05:05.647180+0000","last_clean":"2024-04-25T12:05:05.647180+0000","last_became_active":"2024-04-25T12:05:04.661816+0000","last_became_peered":"2024-04-25T12:05:04.661816+0000","last_unstale":"2024-04-25T12:05:05.647180+0000","last_undegraded":"2024-04-25T12:05:05.647180+0000","last_fullsized":"2024-04-25T12:05:05.647180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:06.656668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.682955+0000","last_change":"2024-04-25T12:05:03.664741+0000","last_active":"2024-04-25T12:05:05.682955+0000","last_peered":"2024-04-25T12:05:05.682955+0000","last_clean":"2024-04-25T12:05:05.682955+0000","last_became_active":"2024-04-25T12:05:03.664373+0000","last_became_peered":"2024-04-25T12:05:03.664373+0000","last_unstale":"2024-04-25T12:05:05.682955+0000","last_undegraded":"2024-04-25T12:05:05.682955+0000","last_fullsized":"2024-04-25T12:05:05.682955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:03.540606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683675+0000","last_change":"2024-04-25T12:05:04.677748+0000","last_active":"2024-04-25T12:05:05.683675+0000","last_peered":"2024-04-25T12:05:05.683675+0000","last_clean":"2024-04-25T12:05:05.683675+0000","last_became_active":"2024-04-25T12:05:04.674003+0000","last_became_peered":"2024-04-25T12:05:04.674003+0000","last_unstale":"2024-04-25T12:05:05.683675+0000","last_undegraded":"2024-04-25T12:05:05.683675+0000","last_fullsized":"2024-04-25T12:05:05.683675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:24.031333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756375+0000","last_change":"2024-04-25T12:05:03.666295+0000","last_active":"2024-04-25T12:05:06.756375+0000","last_peered":"2024-04-25T12:05:06.756375+0000","last_clean":"2024-04-25T12:05:06.756375+0000","last_became_active":"2024-04-25T12:05:03.665910+0000","last_became_peered":"2024-04-25T12:05:03.665910+0000","last_unstale":"2024-04-25T12:05:06.756375+0000","last_undegraded":"2024-04-25T12:05:06.756375+0000","last_fullsized":"2024-04-25T12:05:06.756375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:13.028799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756503+0000","last_change":"2024-04-25T12:05:04.668765+0000","last_active":"2024-04-25T12:05:06.756503+0000","last_peered":"2024-04-25T12:05:06.756503+0000","last_clean":"2024-04-25T12:05:06.756503+0000","last_became_active":"2024-04-25T12:05:04.668391+0000","last_became_peered":"2024-04-25T12:05:04.668391+0000","last_unstale":"2024-04-25T12:05:06.756503+0000","last_undegraded":"2024-04-25T12:05:06.756503+0000","last_fullsized":"2024-04-25T12:05:06.756503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:47.282921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:07.703946+0000","last_change":"2024-04-25T12:05:03.663799+0000","last_active":"2024-04-25T12:05:07.703946+0000","last_peered":"2024-04-25T12:05:07.703946+0000","last_clean":"2024-04-25T12:05:07.703946+0000","last_became_active":"2024-04-25T12:05:03.663501+0000","last_became_peered":"2024-04-25T12:05:03.663501+0000","last_unstale":"2024-04-25T12:05:07.703946+0000","last_undegraded":"2024-04-25T12:05:07.703946+0000","last_fullsized":"2024-04-25T12:05:07.703946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:34.402036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683179+0000","last_change":"2024-04-25T12:05:04.662594+0000","last_active":"2024-04-25T12:05:05.683179+0000","last_peered":"2024-04-25T12:05:05.683179+0000","last_clean":"2024-04-25T12:05:05.683179+0000","last_became_active":"2024-04-25T12:05:04.662179+0000","last_became_peered":"2024-04-25T12:05:04.662179+0000","last_unstale":"2024-04-25T12:05:05.683179+0000","last_undegraded":"2024-04-25T12:05:05.683179+0000","last_fullsized":"2024-04-25T12:05:05.683179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:58.709837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683135+0000","last_change":"2024-04-25T12:05:03.659416+0000","last_active":"2024-04-25T12:05:05.683135+0000","last_peered":"2024-04-25T12:05:05.683135+0000","last_clean":"2024-04-25T12:05:05.683135+0000","last_became_active":"2024-04-25T12:05:03.658416+0000","last_became_peered":"2024-04-25T12:05:03.658416+0000","last_unstale":"2024-04-25T12:05:05.683135+0000","last_undegraded":"2024-04-25T12:05:05.683135+0000","last_fullsized":"2024-04-25T12:05:05.683135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:01.174438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.654872+0000","last_change":"2024-04-25T12:05:04.670890+0000","last_active":"2024-04-25T12:05:05.654872+0000","last_peered":"2024-04-25T12:05:05.654872+0000","last_clean":"2024-04-25T12:05:05.654872+0000","last_became_active":"2024-04-25T12:05:04.670581+0000","last_became_peered":"2024-04-25T12:05:04.670581+0000","last_unstale":"2024-04-25T12:05:05.654872+0000","last_undegraded":"2024-04-25T12:05:05.654872+0000","last_fullsized":"2024-04-25T12:05:05.654872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.527303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:09.720046+0000","last_change":"2024-04-25T12:05:03.662561+0000","last_active":"2024-04-25T12:05:09.720046+0000","last_peered":"2024-04-25T12:05:09.720046+0000","last_clean":"2024-04-25T12:05:09.720046+0000","last_became_active":"2024-04-25T12:05:03.662155+0000","last_became_peered":"2024-04-25T12:05:03.662155+0000","last_unstale":"2024-04-25T12:05:09.720046+0000","last_undegraded":"2024-04-25T12:05:09.720046+0000","last_fullsized":"2024-04-25T12:05:09.720046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:02.130271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683492+0000","last_change":"2024-04-25T12:05:04.669905+0000","last_active":"2024-04-25T12:05:05.683492+0000","last_peered":"2024-04-25T12:05:05.683492+0000","last_clean":"2024-04-25T12:05:05.683492+0000","last_became_active":"2024-04-25T12:05:04.669322+0000","last_became_peered":"2024-04-25T12:05:04.669322+0000","last_unstale":"2024-04-25T12:05:05.683492+0000","last_undegraded":"2024-04-25T12:05:05.683492+0000","last_fullsized":"2024-04-25T12:05:05.683492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.705643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:07.704599+0000","last_change":"2024-04-25T12:05:03.657923+0000","last_active":"2024-04-25T12:05:07.704599+0000","last_peered":"2024-04-25T12:05:07.704599+0000","last_clean":"2024-04-25T12:05:07.704599+0000","last_became_active":"2024-04-25T12:05:03.657332+0000","last_became_peered":"2024-04-25T12:05:03.657332+0000","last_unstale":"2024-04-25T12:05:07.704599+0000","last_undegraded":"2024-04-25T12:05:07.704599+0000","last_fullsized":"2024-04-25T12:05:07.704599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:54.521822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014393+0000","last_change":"2024-04-25T12:05:04.670193+0000","last_active":"2024-04-25T12:05:06.014393+0000","last_peered":"2024-04-25T12:05:06.014393+0000","last_clean":"2024-04-25T12:05:06.014393+0000","last_became_active":"2024-04-25T12:05:04.669720+0000","last_became_peered":"2024-04-25T12:05:04.669720+0000","last_unstale":"2024-04-25T12:05:06.014393+0000","last_undegraded":"2024-04-25T12:05:06.014393+0000","last_fullsized":"2024-04-25T12:05:06.014393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:26.080914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:10.184866+0000","last_change":"2024-04-25T12:05:03.666064+0000","last_active":"2024-04-25T12:05:10.184866+0000","last_peered":"2024-04-25T12:05:10.184866+0000","last_clean":"2024-04-25T12:05:10.184866+0000","last_became_active":"2024-04-25T12:05:03.665749+0000","last_became_peered":"2024-04-25T12:05:03.665749+0000","last_unstale":"2024-04-25T12:05:10.184866+0000","last_undegraded":"2024-04-25T12:05:10.184866+0000","last_fullsized":"2024-04-25T12:05:10.184866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:21.607549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014619+0000","last_change":"2024-04-25T12:05:04.660953+0000","last_active":"2024-04-25T12:05:06.014619+0000","last_peered":"2024-04-25T12:05:06.014619+0000","last_clean":"2024-04-25T12:05:06.014619+0000","last_became_active":"2024-04-25T12:05:04.660577+0000","last_became_peered":"2024-04-25T12:05:04.660577+0000","last_unstale":"2024-04-25T12:05:06.014619+0000","last_undegraded":"2024-04-25T12:05:06.014619+0000","last_fullsized":"2024-04-25T12:05:06.014619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:35.728839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756173+0000","last_change":"2024-04-25T12:05:03.664454+0000","last_active":"2024-04-25T12:05:06.756173+0000","last_peered":"2024-04-25T12:05:06.756173+0000","last_clean":"2024-04-25T12:05:06.756173+0000","last_became_active":"2024-04-25T12:05:03.664064+0000","last_became_peered":"2024-04-25T12:05:03.664064+0000","last_unstale":"2024-04-25T12:05:06.756173+0000","last_undegraded":"2024-04-25T12:05:06.756173+0000","last_fullsized":"2024-04-25T12:05:06.756173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:28.146713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014403+0000","last_change":"2024-04-25T12:05:04.668385+0000","last_active":"2024-04-25T12:05:06.014403+0000","last_peered":"2024-04-25T12:05:06.014403+0000","last_clean":"2024-04-25T12:05:06.014403+0000","last_became_active":"2024-04-25T12:05:04.667995+0000","last_became_peered":"2024-04-25T12:05:04.667995+0000","last_unstale":"2024-04-25T12:05:06.014403+0000","last_undegraded":"2024-04-25T12:05:06.014403+0000","last_fullsized":"2024-04-25T12:05:06.014403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:39.938151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.647278+0000","last_change":"2024-04-25T12:05:03.661795+0000","last_active":"2024-04-25T12:05:05.647278+0000","last_peered":"2024-04-25T12:05:05.647278+0000","last_clean":"2024-04-25T12:05:05.647278+0000","last_became_active":"2024-04-25T12:05:03.661443+0000","last_became_peered":"2024-04-25T12:05:03.661443+0000","last_unstale":"2024-04-25T12:05:05.647278+0000","last_undegraded":"2024-04-25T12:05:05.647278+0000","last_fullsized":"2024-04-25T12:05:05.647278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:52:52.901723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014682+0000","last_change":"2024-04-25T12:05:04.662068+0000","last_active":"2024-04-25T12:05:06.014682+0000","last_peered":"2024-04-25T12:05:06.014682+0000","last_clean":"2024-04-25T12:05:06.014682+0000","last_became_active":"2024-04-25T12:05:04.661694+0000","last_became_peered":"2024-04-25T12:05:04.661694+0000","last_unstale":"2024-04-25T12:05:06.014682+0000","last_undegraded":"2024-04-25T12:05:06.014682+0000","last_fullsized":"2024-04-25T12:05:06.014682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:45.200872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.655010+0000","last_change":"2024-04-25T12:05:04.662320+0000","last_active":"2024-04-25T12:05:05.655010+0000","last_peered":"2024-04-25T12:05:05.655010+0000","last_clean":"2024-04-25T12:05:05.655010+0000","last_became_active":"2024-04-25T12:05:04.661992+0000","last_became_peered":"2024-04-25T12:05:04.661992+0000","last_unstale":"2024-04-25T12:05:05.655010+0000","last_undegraded":"2024-04-25T12:05:05.655010+0000","last_fullsized":"2024-04-25T12:05:05.655010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:25.898506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.650641+0000","last_change":"2024-04-25T12:05:04.671534+0000","last_active":"2024-04-25T12:05:05.650641+0000","last_peered":"2024-04-25T12:05:05.650641+0000","last_clean":"2024-04-25T12:05:05.650641+0000","last_became_active":"2024-04-25T12:05:04.671169+0000","last_became_peered":"2024-04-25T12:05:04.671169+0000","last_unstale":"2024-04-25T12:05:05.650641+0000","last_undegraded":"2024-04-25T12:05:05.650641+0000","last_fullsized":"2024-04-25T12:05:05.650641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:49.972884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.650680+0000","last_change":"2024-04-25T12:05:04.668832+0000","last_active":"2024-04-25T12:05:05.650680+0000","last_peered":"2024-04-25T12:05:05.650680+0000","last_clean":"2024-04-25T12:05:05.650680+0000","last_became_active":"2024-04-25T12:05:04.668459+0000","last_became_peered":"2024-04-25T12:05:04.668459+0000","last_unstale":"2024-04-25T12:05:05.650680+0000","last_undegraded":"2024-04-25T12:05:05.650680+0000","last_fullsized":"2024-04-25T12:05:05.650680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:58.509505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756336+0000","last_change":"2024-04-25T12:05:04.671393+0000","last_active":"2024-04-25T12:05:06.756336+0000","last_peered":"2024-04-25T12:05:06.756336+0000","last_clean":"2024-04-25T12:05:06.756336+0000","last_became_active":"2024-04-25T12:05:04.671041+0000","last_became_peered":"2024-04-25T12:05:04.671041+0000","last_unstale":"2024-04-25T12:05:06.756336+0000","last_undegraded":"2024-04-25T12:05:06.756336+0000","last_fullsized":"2024-04-25T12:05:06.756336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:29.428573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.647368+0000","last_change":"2024-04-25T12:05:04.669936+0000","last_active":"2024-04-25T12:05:05.647368+0000","last_peered":"2024-04-25T12:05:05.647368+0000","last_clean":"2024-04-25T12:05:05.647368+0000","last_became_active":"2024-04-25T12:05:04.669204+0000","last_became_peered":"2024-04-25T12:05:04.669204+0000","last_unstale":"2024-04-25T12:05:05.647368+0000","last_undegraded":"2024-04-25T12:05:05.647368+0000","last_fullsized":"2024-04-25T12:05:05.647368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:38.814562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.655239+0000","last_change":"2024-04-25T12:05:04.664852+0000","last_active":"2024-04-25T12:05:05.655239+0000","last_peered":"2024-04-25T12:05:05.655239+0000","last_clean":"2024-04-25T12:05:05.655239+0000","last_became_active":"2024-04-25T12:05:04.663269+0000","last_became_peered":"2024-04-25T12:05:04.663269+0000","last_unstale":"2024-04-25T12:05:05.655239+0000","last_undegraded":"2024-04-25T12:05:05.655239+0000","last_fullsized":"2024-04-25T12:05:05.655239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:57.862378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.651005+0000","last_change":"2024-04-25T12:05:04.666332+0000","last_active":"2024-04-25T12:05:05.651005+0000","last_peered":"2024-04-25T12:05:05.651005+0000","last_clean":"2024-04-25T12:05:05.651005+0000","last_became_active":"2024-04-25T12:05:04.665777+0000","last_became_peered":"2024-04-25T12:05:04.665777+0000","last_unstale":"2024-04-25T12:05:05.651005+0000","last_undegraded":"2024-04-25T12:05:05.651005+0000","last_fullsized":"2024-04-25T12:05:05.651005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:53.912782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683581+0000","last_change":"2024-04-25T12:05:04.678238+0000","last_active":"2024-04-25T12:05:05.683581+0000","last_peered":"2024-04-25T12:05:05.683581+0000","last_clean":"2024-04-25T12:05:05.683581+0000","last_became_active":"2024-04-25T12:05:04.677660+0000","last_became_peered":"2024-04-25T12:05:04.677660+0000","last_unstale":"2024-04-25T12:05:05.683581+0000","last_undegraded":"2024-04-25T12:05:05.683581+0000","last_fullsized":"2024-04-25T12:05:05.683581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:02.916486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756213+0000","last_change":"2024-04-25T12:05:04.666533+0000","last_active":"2024-04-25T12:05:06.756213+0000","last_peered":"2024-04-25T12:05:06.756213+0000","last_clean":"2024-04-25T12:05:06.756213+0000","last_became_active":"2024-04-25T12:05:04.666184+0000","last_became_peered":"2024-04-25T12:05:04.666184+0000","last_unstale":"2024-04-25T12:05:06.756213+0000","last_undegraded":"2024-04-25T12:05:06.756213+0000","last_fullsized":"2024-04-25T12:05:06.756213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:13.089285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683272+0000","last_change":"2024-04-25T12:05:04.672880+0000","last_active":"2024-04-25T12:05:05.683272+0000","last_peered":"2024-04-25T12:05:05.683272+0000","last_clean":"2024-04-25T12:05:05.683272+0000","last_became_active":"2024-04-25T12:05:04.669640+0000","last_became_peered":"2024-04-25T12:05:04.669640+0000","last_unstale":"2024-04-25T12:05:05.683272+0000","last_undegraded":"2024-04-25T12:05:05.683272+0000","last_fullsized":"2024-04-25T12:05:05.683272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:52.293914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.636078+0000","last_change":"2024-04-25T12:05:04.664274+0000","last_active":"2024-04-25T12:05:06.636078+0000","last_peered":"2024-04-25T12:05:06.636078+0000","last_clean":"2024-04-25T12:05:06.636078+0000","last_became_active":"2024-04-25T12:05:04.663715+0000","last_became_peered":"2024-04-25T12:05:04.663715+0000","last_unstale":"2024-04-25T12:05:06.636078+0000","last_undegraded":"2024-04-25T12:05:06.636078+0000","last_fullsized":"2024-04-25T12:05:06.636078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:28.298147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.647178+0000","last_change":"2024-04-25T12:05:04.669974+0000","last_active":"2024-04-25T12:05:05.647178+0000","last_peered":"2024-04-25T12:05:05.647178+0000","last_clean":"2024-04-25T12:05:05.647178+0000","last_became_active":"2024-04-25T12:05:04.669531+0000","last_became_peered":"2024-04-25T12:05:04.669531+0000","last_unstale":"2024-04-25T12:05:05.647178+0000","last_undegraded":"2024-04-25T12:05:05.647178+0000","last_fullsized":"2024-04-25T12:05:05.647178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:51.579582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.655038+0000","last_change":"2024-04-25T12:05:04.670203+0000","last_active":"2024-04-25T12:05:05.655038+0000","last_peered":"2024-04-25T12:05:05.655038+0000","last_clean":"2024-04-25T12:05:05.655038+0000","last_became_active":"2024-04-25T12:05:04.669865+0000","last_became_peered":"2024-04-25T12:05:04.669865+0000","last_unstale":"2024-04-25T12:05:05.655038+0000","last_undegraded":"2024-04-25T12:05:05.655038+0000","last_fullsized":"2024-04-25T12:05:05.655038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:19.860002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281216,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4952,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":98,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":33557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645330,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":1030695,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070494,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":1030681,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":439215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":461388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":909312,"data_stored":430055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":438543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":1030365,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34127,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34127,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34127,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:05:12.540 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T12:05:12.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.682+0000 7fc7b1407640 1 -- 172.21.15.19:0/2188581004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac105350 msgr2=0x7fc7ac107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.682+0000 7fc7b1407640 1 --2- 172.21.15.19:0/2188581004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac105350 0x7fc7ac107740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009900 tx=0x7fc7a002a410 comp rx=0 tx=0).stop 2024-04-25T12:05:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.682+0000 7fc7b1407640 1 -- 172.21.15.19:0/2188581004 shutdown_connections 2024-04-25T12:05:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.682+0000 7fc7b1407640 1 --2- 172.21.15.19:0/2188581004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7ac107c80 0x7fc7ac10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.682+0000 7fc7b1407640 1 --2- 172.21.15.19:0/2188581004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac105350 0x7fc7ac107740 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.682+0000 7fc7b1407640 1 --2- 172.21.15.19:0/2188581004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7ac102a20 0x7fc7ac104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.682+0000 7fc7b1407640 1 -- 172.21.15.19:0/2188581004 >> 172.21.15.19:0/2188581004 conn(0x7fc7ac0fc430 msgr2=0x7fc7ac0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.682+0000 7fc7b1407640 1 -- 172.21.15.19:0/2188581004 shutdown_connections 2024-04-25T12:05:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.682+0000 7fc7b1407640 1 -- 172.21.15.19:0/2188581004 wait complete. 2024-04-25T12:05:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.683+0000 7fc7b1407640 1 Processor -- start 2024-04-25T12:05:12.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.683+0000 7fc7b1407640 1 -- start start 2024-04-25T12:05:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.683+0000 7fc7b1407640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7ac102a20 0x7fc7ac198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.683+0000 7fc7b1407640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac105350 0x7fc7ac198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.683+0000 7fc7b1407640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7ac107c80 0x7fc7ac19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.683+0000 7fc7b1407640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac10dc00 con 0x7fc7ac105350 2024-04-25T12:05:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.683+0000 7fc7b1407640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac10da80 con 0x7fc7ac102a20 2024-04-25T12:05:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.683+0000 7fc7b1407640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac10dd80 con 0x7fc7ac107c80 2024-04-25T12:05:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.683+0000 7fc7aa7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac105350 0x7fc7ac198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.683+0000 7fc7aa7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac105350 0x7fc7ac198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42716/0 (socket says 172.21.15.19:42716) 2024-04-25T12:05:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.683+0000 7fc7aa7fc640 1 -- 172.21.15.19:0/1868319258 learned_addr learned my addr 172.21.15.19:0/1868319258 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.684+0000 7fc7aa7fc640 1 -- 172.21.15.19:0/1868319258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7ac107c80 msgr2=0x7fc7ac19f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.684+0000 7fc7aa7fc640 1 --2- 172.21.15.19:0/1868319258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7ac107c80 0x7fc7ac19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.684+0000 7fc7aa7fc640 1 -- 172.21.15.19:0/1868319258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7ac102a20 msgr2=0x7fc7ac198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.684+0000 7fc7aa7fc640 1 --2- 172.21.15.19:0/1868319258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7ac102a20 0x7fc7ac198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.684+0000 7fc7aa7fc640 1 -- 172.21.15.19:0/1868319258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac19ff50 con 0x7fc7ac105350 2024-04-25T12:05:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.684+0000 7fc7aa7fc640 1 --2- 172.21.15.19:0/1868319258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac105350 0x7fc7ac198870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0006010 tx=0x7fc7a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.684+0000 7fc793fff640 1 -- 172.21.15.19:0/1868319258 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a003f030 con 0x7fc7ac105350 2024-04-25T12:05:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.684+0000 7fc793fff640 1 -- 172.21.15.19:0/1868319258 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc7a0003cd0 con 0x7fc7ac105350 2024-04-25T12:05:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.684+0000 7fc7b1407640 1 -- 172.21.15.19:0/1868319258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ac1a01e0 con 0x7fc7ac105350 2024-04-25T12:05:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.684+0000 7fc793fff640 1 -- 172.21.15.19:0/1868319258 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0035070 con 0x7fc7ac105350 2024-04-25T12:05:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.684+0000 7fc7b1407640 1 -- 172.21.15.19:0/1868319258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ac1a06d0 con 0x7fc7ac105350 2024-04-25T12:05:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.685+0000 7fc7b1407640 1 -- 172.21.15.19:0/1868319258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ac0fdf20 con 0x7fc7ac105350 2024-04-25T12:05:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.686+0000 7fc793fff640 1 -- 172.21.15.19:0/1868319258 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc7a0042030 con 0x7fc7ac105350 2024-04-25T12:05:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.687+0000 7fc793fff640 1 --2- 172.21.15.19:0/1868319258 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc7940a3c40 0x7fc7940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.687+0000 7fc7aaffd640 1 --2- 172.21.15.19:0/1868319258 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc7940a3c40 0x7fc7940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.687+0000 7fc793fff640 1 -- 172.21.15.19:0/1868319258 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fc7a00f0e70 con 0x7fc7ac105350 2024-04-25T12:05:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.687+0000 7fc7aaffd640 1 --2- 172.21.15.19:0/1868319258 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc7940a3c40 0x7fc7940a6100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc79c005c40 tx=0x7fc79c005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.691+0000 7fc793fff640 1 -- 172.21.15.19:0/1868319258 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7a00bd820 con 0x7fc7ac105350 2024-04-25T12:05:12.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:12 smithi019 ceph-mon[29213]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:12.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:12 smithi019 ceph-mon[29213]: pgmap v293: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 564 B/s rd, 4.0 KiB/s wr, 14 op/s 2024-04-25T12:05:12.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:12 smithi019 ceph-mon[29213]: from='client.24625 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:12.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:12 smithi019 ceph-mon[29213]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:12.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:12 smithi019 ceph-mon[29213]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T12:05:12.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:12 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2290166817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:05:12.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:12 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1451737908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:12.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.875+0000 7fc7b1407640 1 -- 172.21.15.19:0/1868319258 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7ac062390 con 0x7fc7940a3c40 2024-04-25T12:05:12.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.882+0000 7fc793fff640 1 -- 172.21.15.19:0/1868319258 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258898 (secure 0 0 0) 0x7fc7ac062390 con 0x7fc7940a3c40 2024-04-25T12:05:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:05:12.883 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T12:05:12.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.885+0000 7fc7b1407640 1 -- 172.21.15.19:0/1868319258 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc7940a3c40 msgr2=0x7fc7940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.885+0000 7fc7b1407640 1 --2- 172.21.15.19:0/1868319258 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc7940a3c40 0x7fc7940a6100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc79c005c40 tx=0x7fc79c005bd0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.885+0000 7fc7b1407640 1 -- 172.21.15.19:0/1868319258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac105350 msgr2=0x7fc7ac198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:12.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.885+0000 7fc7b1407640 1 --2- 172.21.15.19:0/1868319258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac105350 0x7fc7ac198870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0006010 tx=0x7fc7a00042f0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.885+0000 7fc7b1407640 1 -- 172.21.15.19:0/1868319258 shutdown_connections 2024-04-25T12:05:12.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.885+0000 7fc7b1407640 1 --2- 172.21.15.19:0/1868319258 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc7940a3c40 0x7fc7940a6100 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.885+0000 7fc7b1407640 1 --2- 172.21.15.19:0/1868319258 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7ac107c80 0x7fc7ac19f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.885+0000 7fc7b1407640 1 --2- 172.21.15.19:0/1868319258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ac105350 0x7fc7ac198870 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.885+0000 7fc7b1407640 1 --2- 172.21.15.19:0/1868319258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7ac102a20 0x7fc7ac198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:12.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.885+0000 7fc7b1407640 1 -- 172.21.15.19:0/1868319258 >> 172.21.15.19:0/1868319258 conn(0x7fc7ac0fc430 msgr2=0x7fc7ac1034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:12.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.885+0000 7fc7b1407640 1 -- 172.21.15.19:0/1868319258 shutdown_connections 2024-04-25T12:05:12.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:12.885+0000 7fc7b1407640 1 -- 172.21.15.19:0/1868319258 wait complete. 2024-04-25T12:05:12.894 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-04-25T12:05:10.896634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595320,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":141,"num_read_kb":123,"num_write":279,"num_write_kb":2967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":239,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364380,"kb_used_data":11028,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577444,"statfs":{"total":1151940427776,"available":1151567302656,"internally_reserved":0,"allocated":11292672,"data_stored":7057061,"data_compressed":107301,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4952,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":98,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.255929"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.655237+0000","last_change":"2024-04-25T12:05:04.662228+0000","last_active":"2024-04-25T12:05:05.655237+0000","last_peered":"2024-04-25T12:05:05.655237+0000","last_clean":"2024-04-25T12:05:05.655237+0000","last_became_active":"2024-04-25T12:05:04.661855+0000","last_became_peered":"2024-04-25T12:05:04.661855+0000","last_unstale":"2024-04-25T12:05:05.655237+0000","last_undegraded":"2024-04-25T12:05:05.655237+0000","last_fullsized":"2024-04-25T12:05:05.655237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:41.662330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756444+0000","last_change":"2024-04-25T12:05:04.664103+0000","last_active":"2024-04-25T12:05:06.756444+0000","last_peered":"2024-04-25T12:05:06.756444+0000","last_clean":"2024-04-25T12:05:06.756444+0000","last_became_active":"2024-04-25T12:05:04.661946+0000","last_became_peered":"2024-04-25T12:05:04.661946+0000","last_unstale":"2024-04-25T12:05:06.756444+0000","last_undegraded":"2024-04-25T12:05:06.756444+0000","last_fullsized":"2024-04-25T12:05:06.756444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:08.119278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756352+0000","last_change":"2024-04-25T12:05:04.667608+0000","last_active":"2024-04-25T12:05:06.756352+0000","last_peered":"2024-04-25T12:05:06.756352+0000","last_clean":"2024-04-25T12:05:06.756352+0000","last_became_active":"2024-04-25T12:05:04.667227+0000","last_became_peered":"2024-04-25T12:05:04.667227+0000","last_unstale":"2024-04-25T12:05:06.756352+0000","last_undegraded":"2024-04-25T12:05:06.756352+0000","last_fullsized":"2024-04-25T12:05:06.756352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:17.794368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756489+0000","last_change":"2024-04-25T12:05:04.671431+0000","last_active":"2024-04-25T12:05:06.756489+0000","last_peered":"2024-04-25T12:05:06.756489+0000","last_clean":"2024-04-25T12:05:06.756489+0000","last_became_active":"2024-04-25T12:05:04.670876+0000","last_became_peered":"2024-04-25T12:05:04.670876+0000","last_unstale":"2024-04-25T12:05:06.756489+0000","last_undegraded":"2024-04-25T12:05:06.756489+0000","last_fullsized":"2024-04-25T12:05:06.756489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:03.564092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683251+0000","last_change":"2024-04-25T12:05:04.665061+0000","last_active":"2024-04-25T12:05:05.683251+0000","last_peered":"2024-04-25T12:05:05.683251+0000","last_clean":"2024-04-25T12:05:05.683251+0000","last_became_active":"2024-04-25T12:05:04.664705+0000","last_became_peered":"2024-04-25T12:05:04.664705+0000","last_unstale":"2024-04-25T12:05:05.683251+0000","last_undegraded":"2024-04-25T12:05:05.683251+0000","last_fullsized":"2024-04-25T12:05:05.683251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:16.434402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.648451+0000","last_change":"2024-04-25T12:05:04.668285+0000","last_active":"2024-04-25T12:05:05.648451+0000","last_peered":"2024-04-25T12:05:05.648451+0000","last_clean":"2024-04-25T12:05:05.648451+0000","last_became_active":"2024-04-25T12:05:04.667941+0000","last_became_peered":"2024-04-25T12:05:04.667941+0000","last_unstale":"2024-04-25T12:05:05.648451+0000","last_undegraded":"2024-04-25T12:05:05.648451+0000","last_fullsized":"2024-04-25T12:05:05.648451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:02.527280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014139+0000","last_change":"2024-04-25T12:05:04.661283+0000","last_active":"2024-04-25T12:05:06.014139+0000","last_peered":"2024-04-25T12:05:06.014139+0000","last_clean":"2024-04-25T12:05:06.014139+0000","last_became_active":"2024-04-25T12:05:04.660944+0000","last_became_peered":"2024-04-25T12:05:04.660944+0000","last_unstale":"2024-04-25T12:05:06.014139+0000","last_undegraded":"2024-04-25T12:05:06.014139+0000","last_fullsized":"2024-04-25T12:05:06.014139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:37.286135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683124+0000","last_change":"2024-04-25T12:05:04.672998+0000","last_active":"2024-04-25T12:05:05.683124+0000","last_peered":"2024-04-25T12:05:05.683124+0000","last_clean":"2024-04-25T12:05:05.683124+0000","last_became_active":"2024-04-25T12:05:04.672611+0000","last_became_peered":"2024-04-25T12:05:04.672611+0000","last_unstale":"2024-04-25T12:05:05.683124+0000","last_undegraded":"2024-04-25T12:05:05.683124+0000","last_fullsized":"2024-04-25T12:05:05.683124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:22.926657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.636465+0000","last_change":"2024-04-25T12:05:04.662957+0000","last_active":"2024-04-25T12:05:06.636465+0000","last_peered":"2024-04-25T12:05:06.636465+0000","last_clean":"2024-04-25T12:05:06.636465+0000","last_became_active":"2024-04-25T12:05:04.662636+0000","last_became_peered":"2024-04-25T12:05:04.662636+0000","last_unstale":"2024-04-25T12:05:06.636465+0000","last_undegraded":"2024-04-25T12:05:06.636465+0000","last_fullsized":"2024-04-25T12:05:06.636465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:05.209057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.648285+0000","last_change":"2024-04-25T12:05:04.670076+0000","last_active":"2024-04-25T12:05:05.648285+0000","last_peered":"2024-04-25T12:05:05.648285+0000","last_clean":"2024-04-25T12:05:05.648285+0000","last_became_active":"2024-04-25T12:05:04.669659+0000","last_became_peered":"2024-04-25T12:05:04.669659+0000","last_unstale":"2024-04-25T12:05:05.648285+0000","last_undegraded":"2024-04-25T12:05:05.648285+0000","last_fullsized":"2024-04-25T12:05:05.648285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:06.503269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014504+0000","last_change":"2024-04-25T12:05:04.664849+0000","last_active":"2024-04-25T12:05:06.014504+0000","last_peered":"2024-04-25T12:05:06.014504+0000","last_clean":"2024-04-25T12:05:06.014504+0000","last_became_active":"2024-04-25T12:05:04.664322+0000","last_became_peered":"2024-04-25T12:05:04.664322+0000","last_unstale":"2024-04-25T12:05:06.014504+0000","last_undegraded":"2024-04-25T12:05:06.014504+0000","last_fullsized":"2024-04-25T12:05:06.014504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:59:03.872076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756448+0000","last_change":"2024-04-25T12:05:04.669403+0000","last_active":"2024-04-25T12:05:06.756448+0000","last_peered":"2024-04-25T12:05:06.756448+0000","last_clean":"2024-04-25T12:05:06.756448+0000","last_became_active":"2024-04-25T12:05:04.669052+0000","last_became_peered":"2024-04-25T12:05:04.669052+0000","last_unstale":"2024-04-25T12:05:06.756448+0000","last_undegraded":"2024-04-25T12:05:06.756448+0000","last_fullsized":"2024-04-25T12:05:06.756448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:18.997019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.647243+0000","last_change":"2024-04-25T12:05:04.664312+0000","last_active":"2024-04-25T12:05:05.647243+0000","last_peered":"2024-04-25T12:05:05.647243+0000","last_clean":"2024-04-25T12:05:05.647243+0000","last_became_active":"2024-04-25T12:05:04.663980+0000","last_became_peered":"2024-04-25T12:05:04.663980+0000","last_unstale":"2024-04-25T12:05:05.647243+0000","last_undegraded":"2024-04-25T12:05:05.647243+0000","last_fullsized":"2024-04-25T12:05:05.647243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:58.427231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756149+0000","last_change":"2024-04-25T12:05:04.663328+0000","last_active":"2024-04-25T12:05:06.756149+0000","last_peered":"2024-04-25T12:05:06.756149+0000","last_clean":"2024-04-25T12:05:06.756149+0000","last_became_active":"2024-04-25T12:05:04.662575+0000","last_became_peered":"2024-04-25T12:05:04.662575+0000","last_unstale":"2024-04-25T12:05:06.756149+0000","last_undegraded":"2024-04-25T12:05:06.756149+0000","last_fullsized":"2024-04-25T12:05:06.756149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:41.786213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756095+0000","last_change":"2024-04-25T12:05:04.672332+0000","last_active":"2024-04-25T12:05:06.756095+0000","last_peered":"2024-04-25T12:05:06.756095+0000","last_clean":"2024-04-25T12:05:06.756095+0000","last_became_active":"2024-04-25T12:05:04.671951+0000","last_became_peered":"2024-04-25T12:05:04.671951+0000","last_unstale":"2024-04-25T12:05:06.756095+0000","last_undegraded":"2024-04-25T12:05:06.756095+0000","last_fullsized":"2024-04-25T12:05:06.756095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:27.283686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.635952+0000","last_change":"2024-04-25T12:05:04.669224+0000","last_active":"2024-04-25T12:05:06.635952+0000","last_peered":"2024-04-25T12:05:06.635952+0000","last_clean":"2024-04-25T12:05:06.635952+0000","last_became_active":"2024-04-25T12:05:04.668894+0000","last_became_peered":"2024-04-25T12:05:04.668894+0000","last_unstale":"2024-04-25T12:05:06.635952+0000","last_undegraded":"2024-04-25T12:05:06.635952+0000","last_fullsized":"2024-04-25T12:05:06.635952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:19.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.655361+0000","last_change":"2024-04-25T12:05:04.670391+0000","last_active":"2024-04-25T12:05:05.655361+0000","last_peered":"2024-04-25T12:05:05.655361+0000","last_clean":"2024-04-25T12:05:05.655361+0000","last_became_active":"2024-04-25T12:05:04.670021+0000","last_became_peered":"2024-04-25T12:05:04.670021+0000","last_unstale":"2024-04-25T12:05:05.655361+0000","last_undegraded":"2024-04-25T12:05:05.655361+0000","last_fullsized":"2024-04-25T12:05:05.655361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:40.214491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.681536+0000","last_change":"2024-04-25T12:05:04.665102+0000","last_active":"2024-04-25T12:05:05.681536+0000","last_peered":"2024-04-25T12:05:05.681536+0000","last_clean":"2024-04-25T12:05:05.681536+0000","last_became_active":"2024-04-25T12:05:04.664719+0000","last_became_peered":"2024-04-25T12:05:04.664719+0000","last_unstale":"2024-04-25T12:05:05.681536+0000","last_undegraded":"2024-04-25T12:05:05.681536+0000","last_fullsized":"2024-04-25T12:05:05.681536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:37.528031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.650833+0000","last_change":"2024-04-25T12:05:04.667931+0000","last_active":"2024-04-25T12:05:05.650833+0000","last_peered":"2024-04-25T12:05:05.650833+0000","last_clean":"2024-04-25T12:05:05.650833+0000","last_became_active":"2024-04-25T12:05:04.667606+0000","last_became_peered":"2024-04-25T12:05:04.667606+0000","last_unstale":"2024-04-25T12:05:05.650833+0000","last_undegraded":"2024-04-25T12:05:05.650833+0000","last_fullsized":"2024-04-25T12:05:05.650833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:43.265302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.681476+0000","last_change":"2024-04-25T12:05:03.656983+0000","last_active":"2024-04-25T12:05:05.681476+0000","last_peered":"2024-04-25T12:05:05.681476+0000","last_clean":"2024-04-25T12:05:05.681476+0000","last_became_active":"2024-04-25T12:05:03.656596+0000","last_became_peered":"2024-04-25T12:05:03.656596+0000","last_unstale":"2024-04-25T12:05:05.681476+0000","last_undegraded":"2024-04-25T12:05:05.681476+0000","last_fullsized":"2024-04-25T12:05:05.681476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:25.133153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683650+0000","last_change":"2024-04-25T12:05:04.668880+0000","last_active":"2024-04-25T12:05:05.683650+0000","last_peered":"2024-04-25T12:05:05.683650+0000","last_clean":"2024-04-25T12:05:05.683650+0000","last_became_active":"2024-04-25T12:05:04.668463+0000","last_became_peered":"2024-04-25T12:05:04.668463+0000","last_unstale":"2024-04-25T12:05:05.683650+0000","last_undegraded":"2024-04-25T12:05:05.683650+0000","last_fullsized":"2024-04-25T12:05:05.683650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:52.647241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683697+0000","last_change":"2024-04-25T12:05:03.663928+0000","last_active":"2024-04-25T12:05:05.683697+0000","last_peered":"2024-04-25T12:05:05.683697+0000","last_clean":"2024-04-25T12:05:05.683697+0000","last_became_active":"2024-04-25T12:05:03.663611+0000","last_became_peered":"2024-04-25T12:05:03.663611+0000","last_unstale":"2024-04-25T12:05:05.683697+0000","last_undegraded":"2024-04-25T12:05:05.683697+0000","last_fullsized":"2024-04-25T12:05:05.683697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:26.986245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756710+0000","last_change":"2024-04-25T12:05:04.674426+0000","last_active":"2024-04-25T12:05:06.756710+0000","last_peered":"2024-04-25T12:05:06.756710+0000","last_clean":"2024-04-25T12:05:06.756710+0000","last_became_active":"2024-04-25T12:05:04.668668+0000","last_became_peered":"2024-04-25T12:05:04.668668+0000","last_unstale":"2024-04-25T12:05:06.756710+0000","last_undegraded":"2024-04-25T12:05:06.756710+0000","last_fullsized":"2024-04-25T12:05:06.756710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:34.205038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:09.719974+0000","last_change":"2024-04-25T12:05:03.656464+0000","last_active":"2024-04-25T12:05:09.719974+0000","last_peered":"2024-04-25T12:05:09.719974+0000","last_clean":"2024-04-25T12:05:09.719974+0000","last_became_active":"2024-04-25T12:05:03.656109+0000","last_became_peered":"2024-04-25T12:05:03.656109+0000","last_unstale":"2024-04-25T12:05:09.719974+0000","last_undegraded":"2024-04-25T12:05:09.719974+0000","last_fullsized":"2024-04-25T12:05:09.719974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:32.175997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.636147+0000","last_change":"2024-04-25T12:05:04.662444+0000","last_active":"2024-04-25T12:05:06.636147+0000","last_peered":"2024-04-25T12:05:06.636147+0000","last_clean":"2024-04-25T12:05:06.636147+0000","last_became_active":"2024-04-25T12:05:04.659285+0000","last_became_peered":"2024-04-25T12:05:04.659285+0000","last_unstale":"2024-04-25T12:05:06.636147+0000","last_undegraded":"2024-04-25T12:05:06.636147+0000","last_fullsized":"2024-04-25T12:05:06.636147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:32.678020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683481+0000","last_change":"2024-04-25T12:05:03.656911+0000","last_active":"2024-04-25T12:05:05.683481+0000","last_peered":"2024-04-25T12:05:05.683481+0000","last_clean":"2024-04-25T12:05:05.683481+0000","last_became_active":"2024-04-25T12:05:03.656553+0000","last_became_peered":"2024-04-25T12:05:03.656553+0000","last_unstale":"2024-04-25T12:05:05.683481+0000","last_undegraded":"2024-04-25T12:05:05.683481+0000","last_fullsized":"2024-04-25T12:05:05.683481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:24.154910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756500+0000","last_change":"2024-04-25T12:05:04.674119+0000","last_active":"2024-04-25T12:05:06.756500+0000","last_peered":"2024-04-25T12:05:06.756500+0000","last_clean":"2024-04-25T12:05:06.756500+0000","last_became_active":"2024-04-25T12:05:04.673260+0000","last_became_peered":"2024-04-25T12:05:04.673260+0000","last_unstale":"2024-04-25T12:05:06.756500+0000","last_undegraded":"2024-04-25T12:05:06.756500+0000","last_fullsized":"2024-04-25T12:05:06.756500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:24.941232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:07.703875+0000","last_change":"2024-04-25T12:05:03.658259+0000","last_active":"2024-04-25T12:05:07.703875+0000","last_peered":"2024-04-25T12:05:07.703875+0000","last_clean":"2024-04-25T12:05:07.703875+0000","last_became_active":"2024-04-25T12:05:03.657917+0000","last_became_peered":"2024-04-25T12:05:03.657917+0000","last_unstale":"2024-04-25T12:05:07.703875+0000","last_undegraded":"2024-04-25T12:05:07.703875+0000","last_fullsized":"2024-04-25T12:05:07.703875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:17.780674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683341+0000","last_change":"2024-04-25T12:05:04.657620+0000","last_active":"2024-04-25T12:05:05.683341+0000","last_peered":"2024-04-25T12:05:05.683341+0000","last_clean":"2024-04-25T12:05:05.683341+0000","last_became_active":"2024-04-25T12:05:04.657264+0000","last_became_peered":"2024-04-25T12:05:04.657264+0000","last_unstale":"2024-04-25T12:05:05.683341+0000","last_undegraded":"2024-04-25T12:05:05.683341+0000","last_fullsized":"2024-04-25T12:05:05.683341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:08.159694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.648192+0000","last_change":"2024-04-25T12:05:03.662345+0000","last_active":"2024-04-25T12:05:05.648192+0000","last_peered":"2024-04-25T12:05:05.648192+0000","last_clean":"2024-04-25T12:05:05.648192+0000","last_became_active":"2024-04-25T12:05:03.661843+0000","last_became_peered":"2024-04-25T12:05:03.661843+0000","last_unstale":"2024-04-25T12:05:05.648192+0000","last_undegraded":"2024-04-25T12:05:05.648192+0000","last_fullsized":"2024-04-25T12:05:05.648192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:03.904192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756488+0000","last_change":"2024-04-25T12:05:04.674954+0000","last_active":"2024-04-25T12:05:06.756488+0000","last_peered":"2024-04-25T12:05:06.756488+0000","last_clean":"2024-04-25T12:05:06.756488+0000","last_became_active":"2024-04-25T12:05:04.673652+0000","last_became_peered":"2024-04-25T12:05:04.673652+0000","last_unstale":"2024-04-25T12:05:06.756488+0000","last_undegraded":"2024-04-25T12:05:06.756488+0000","last_fullsized":"2024-04-25T12:05:06.756488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:26.166918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.647011+0000","last_change":"2024-04-25T12:05:03.656638+0000","last_active":"2024-04-25T12:05:05.647011+0000","last_peered":"2024-04-25T12:05:05.647011+0000","last_clean":"2024-04-25T12:05:05.647011+0000","last_became_active":"2024-04-25T12:05:03.656276+0000","last_became_peered":"2024-04-25T12:05:03.656276+0000","last_unstale":"2024-04-25T12:05:05.647011+0000","last_undegraded":"2024-04-25T12:05:05.647011+0000","last_fullsized":"2024-04-25T12:05:05.647011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:30.034633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683036+0000","last_change":"2024-04-25T12:05:04.674146+0000","last_active":"2024-04-25T12:05:05.683036+0000","last_peered":"2024-04-25T12:05:05.683036+0000","last_clean":"2024-04-25T12:05:05.683036+0000","last_became_active":"2024-04-25T12:05:04.673763+0000","last_became_peered":"2024-04-25T12:05:04.673763+0000","last_unstale":"2024-04-25T12:05:05.683036+0000","last_undegraded":"2024-04-25T12:05:05.683036+0000","last_fullsized":"2024-04-25T12:05:05.683036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:09.121546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.755972+0000","last_change":"2024-04-25T12:05:03.658527+0000","last_active":"2024-04-25T12:05:06.755972+0000","last_peered":"2024-04-25T12:05:06.755972+0000","last_clean":"2024-04-25T12:05:06.755972+0000","last_became_active":"2024-04-25T12:05:03.658200+0000","last_became_peered":"2024-04-25T12:05:03.658200+0000","last_unstale":"2024-04-25T12:05:06.755972+0000","last_undegraded":"2024-04-25T12:05:06.755972+0000","last_fullsized":"2024-04-25T12:05:06.755972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:17.367641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.681376+0000","last_change":"2024-04-25T12:05:04.666431+0000","last_active":"2024-04-25T12:05:05.681376+0000","last_peered":"2024-04-25T12:05:05.681376+0000","last_clean":"2024-04-25T12:05:05.681376+0000","last_became_active":"2024-04-25T12:05:04.666012+0000","last_became_peered":"2024-04-25T12:05:04.666012+0000","last_unstale":"2024-04-25T12:05:05.681376+0000","last_undegraded":"2024-04-25T12:05:05.681376+0000","last_fullsized":"2024-04-25T12:05:05.681376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:26.183735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.648144+0000","last_change":"2024-04-25T12:05:03.659730+0000","last_active":"2024-04-25T12:05:05.648144+0000","last_peered":"2024-04-25T12:05:05.648144+0000","last_clean":"2024-04-25T12:05:05.648144+0000","last_became_active":"2024-04-25T12:05:03.659359+0000","last_became_peered":"2024-04-25T12:05:03.659359+0000","last_unstale":"2024-04-25T12:05:05.648144+0000","last_undegraded":"2024-04-25T12:05:05.648144+0000","last_fullsized":"2024-04-25T12:05:05.648144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:16.644771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.636458+0000","last_change":"2024-04-25T12:05:04.666603+0000","last_active":"2024-04-25T12:05:06.636458+0000","last_peered":"2024-04-25T12:05:06.636458+0000","last_clean":"2024-04-25T12:05:06.636458+0000","last_became_active":"2024-04-25T12:05:04.666129+0000","last_became_peered":"2024-04-25T12:05:04.666129+0000","last_unstale":"2024-04-25T12:05:06.636458+0000","last_undegraded":"2024-04-25T12:05:06.636458+0000","last_fullsized":"2024-04-25T12:05:06.636458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:01.024544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:07.703054+0000","last_change":"2024-04-25T12:05:03.660353+0000","last_active":"2024-04-25T12:05:07.703054+0000","last_peered":"2024-04-25T12:05:07.703054+0000","last_clean":"2024-04-25T12:05:07.703054+0000","last_became_active":"2024-04-25T12:05:03.659946+0000","last_became_peered":"2024-04-25T12:05:03.659946+0000","last_unstale":"2024-04-25T12:05:07.703054+0000","last_undegraded":"2024-04-25T12:05:07.703054+0000","last_fullsized":"2024-04-25T12:05:07.703054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:47.492835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756609+0000","last_change":"2024-04-25T12:05:04.669930+0000","last_active":"2024-04-25T12:05:06.756609+0000","last_peered":"2024-04-25T12:05:06.756609+0000","last_clean":"2024-04-25T12:05:06.756609+0000","last_became_active":"2024-04-25T12:05:04.669584+0000","last_became_peered":"2024-04-25T12:05:04.669584+0000","last_unstale":"2024-04-25T12:05:06.756609+0000","last_undegraded":"2024-04-25T12:05:06.756609+0000","last_fullsized":"2024-04-25T12:05:06.756609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:47.250335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756478+0000","last_change":"2024-04-25T12:05:03.661346+0000","last_active":"2024-04-25T12:05:06.756478+0000","last_peered":"2024-04-25T12:05:06.756478+0000","last_clean":"2024-04-25T12:05:06.756478+0000","last_became_active":"2024-04-25T12:05:03.661023+0000","last_became_peered":"2024-04-25T12:05:03.661023+0000","last_unstale":"2024-04-25T12:05:06.756478+0000","last_undegraded":"2024-04-25T12:05:06.756478+0000","last_fullsized":"2024-04-25T12:05:06.756478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:56.659194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.636278+0000","last_change":"2024-04-25T12:05:04.668589+0000","last_active":"2024-04-25T12:05:06.636278+0000","last_peered":"2024-04-25T12:05:06.636278+0000","last_clean":"2024-04-25T12:05:06.636278+0000","last_became_active":"2024-04-25T12:05:04.668124+0000","last_became_peered":"2024-04-25T12:05:04.668124+0000","last_unstale":"2024-04-25T12:05:06.636278+0000","last_undegraded":"2024-04-25T12:05:06.636278+0000","last_fullsized":"2024-04-25T12:05:06.636278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:35.878775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.648326+0000","last_change":"2024-04-25T12:05:03.656287+0000","last_active":"2024-04-25T12:05:05.648326+0000","last_peered":"2024-04-25T12:05:05.648326+0000","last_clean":"2024-04-25T12:05:05.648326+0000","last_became_active":"2024-04-25T12:05:03.655928+0000","last_became_peered":"2024-04-25T12:05:03.655928+0000","last_unstale":"2024-04-25T12:05:05.648326+0000","last_undegraded":"2024-04-25T12:05:05.648326+0000","last_fullsized":"2024-04-25T12:05:05.648326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:53.260268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683564+0000","last_change":"2024-04-25T12:05:04.662866+0000","last_active":"2024-04-25T12:05:05.683564+0000","last_peered":"2024-04-25T12:05:05.683564+0000","last_clean":"2024-04-25T12:05:05.683564+0000","last_became_active":"2024-04-25T12:05:04.661181+0000","last_became_peered":"2024-04-25T12:05:04.661181+0000","last_unstale":"2024-04-25T12:05:05.683564+0000","last_undegraded":"2024-04-25T12:05:05.683564+0000","last_fullsized":"2024-04-25T12:05:05.683564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:46.630686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:10.192473+0000","last_change":"2024-04-25T12:05:03.663558+0000","last_active":"2024-04-25T12:05:10.192473+0000","last_peered":"2024-04-25T12:05:10.192473+0000","last_clean":"2024-04-25T12:05:10.192473+0000","last_became_active":"2024-04-25T12:05:03.661650+0000","last_became_peered":"2024-04-25T12:05:03.661650+0000","last_unstale":"2024-04-25T12:05:10.192473+0000","last_undegraded":"2024-04-25T12:05:10.192473+0000","last_fullsized":"2024-04-25T12:05:10.192473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:27.599992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.647103+0000","last_change":"2024-04-25T12:05:04.661283+0000","last_active":"2024-04-25T12:05:05.647103+0000","last_peered":"2024-04-25T12:05:05.647103+0000","last_clean":"2024-04-25T12:05:05.647103+0000","last_became_active":"2024-04-25T12:05:04.660900+0000","last_became_peered":"2024-04-25T12:05:04.660900+0000","last_unstale":"2024-04-25T12:05:05.647103+0000","last_undegraded":"2024-04-25T12:05:05.647103+0000","last_fullsized":"2024-04-25T12:05:05.647103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:56.670962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:07.703488+0000","last_change":"2024-04-25T12:05:03.662922+0000","last_active":"2024-04-25T12:05:07.703488+0000","last_peered":"2024-04-25T12:05:07.703488+0000","last_clean":"2024-04-25T12:05:07.703488+0000","last_became_active":"2024-04-25T12:05:03.662584+0000","last_became_peered":"2024-04-25T12:05:03.662584+0000","last_unstale":"2024-04-25T12:05:07.703488+0000","last_undegraded":"2024-04-25T12:05:07.703488+0000","last_fullsized":"2024-04-25T12:05:07.703488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:41.664030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014495+0000","last_change":"2024-04-25T12:05:04.672018+0000","last_active":"2024-04-25T12:05:06.014495+0000","last_peered":"2024-04-25T12:05:06.014495+0000","last_clean":"2024-04-25T12:05:06.014495+0000","last_became_active":"2024-04-25T12:05:04.671670+0000","last_became_peered":"2024-04-25T12:05:04.671670+0000","last_unstale":"2024-04-25T12:05:06.014495+0000","last_undegraded":"2024-04-25T12:05:06.014495+0000","last_fullsized":"2024-04-25T12:05:06.014495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:45.005941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756324+0000","last_change":"2024-04-25T12:05:03.656826+0000","last_active":"2024-04-25T12:05:06.756324+0000","last_peered":"2024-04-25T12:05:06.756324+0000","last_clean":"2024-04-25T12:05:06.756324+0000","last_became_active":"2024-04-25T12:05:03.656477+0000","last_became_peered":"2024-04-25T12:05:03.656477+0000","last_unstale":"2024-04-25T12:05:06.756324+0000","last_undegraded":"2024-04-25T12:05:06.756324+0000","last_fullsized":"2024-04-25T12:05:06.756324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:38.111569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014098+0000","last_change":"2024-04-25T12:05:04.671771+0000","last_active":"2024-04-25T12:05:06.014098+0000","last_peered":"2024-04-25T12:05:06.014098+0000","last_clean":"2024-04-25T12:05:06.014098+0000","last_became_active":"2024-04-25T12:05:04.671424+0000","last_became_peered":"2024-04-25T12:05:04.671424+0000","last_unstale":"2024-04-25T12:05:06.014098+0000","last_undegraded":"2024-04-25T12:05:06.014098+0000","last_fullsized":"2024-04-25T12:05:06.014098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:51.327337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'16","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:08.709829+0000","last_change":"2024-04-25T12:05:03.655884+0000","last_active":"2024-04-25T12:05:08.709829+0000","last_peered":"2024-04-25T12:05:08.709829+0000","last_clean":"2024-04-25T12:05:08.709829+0000","last_became_active":"2024-04-25T12:05:03.655437+0000","last_became_peered":"2024-04-25T12:05:03.655437+0000","last_unstale":"2024-04-25T12:05:08.709829+0000","last_undegraded":"2024-04-25T12:05:08.709829+0000","last_fullsized":"2024-04-25T12:05:08.709829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:24.497861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756350+0000","last_change":"2024-04-25T12:05:04.674785+0000","last_active":"2024-04-25T12:05:06.756350+0000","last_peered":"2024-04-25T12:05:06.756350+0000","last_clean":"2024-04-25T12:05:06.756350+0000","last_became_active":"2024-04-25T12:05:04.674469+0000","last_became_peered":"2024-04-25T12:05:04.674469+0000","last_unstale":"2024-04-25T12:05:06.756350+0000","last_undegraded":"2024-04-25T12:05:06.756350+0000","last_fullsized":"2024-04-25T12:05:06.756350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:28.616188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683633+0000","last_change":"2024-04-25T12:05:03.653954+0000","last_active":"2024-04-25T12:05:05.683633+0000","last_peered":"2024-04-25T12:05:05.683633+0000","last_clean":"2024-04-25T12:05:05.683633+0000","last_became_active":"2024-04-25T12:05:03.653583+0000","last_became_peered":"2024-04-25T12:05:03.653583+0000","last_unstale":"2024-04-25T12:05:05.683633+0000","last_undegraded":"2024-04-25T12:05:05.683633+0000","last_fullsized":"2024-04-25T12:05:05.683633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:58.007741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.655154+0000","last_change":"2024-04-25T12:05:04.669395+0000","last_active":"2024-04-25T12:05:05.655154+0000","last_peered":"2024-04-25T12:05:05.655154+0000","last_clean":"2024-04-25T12:05:05.655154+0000","last_became_active":"2024-04-25T12:05:04.669058+0000","last_became_peered":"2024-04-25T12:05:04.669058+0000","last_unstale":"2024-04-25T12:05:05.655154+0000","last_undegraded":"2024-04-25T12:05:05.655154+0000","last_fullsized":"2024-04-25T12:05:05.655154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:03.195889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.681359+0000","last_change":"2024-04-25T12:05:03.658899+0000","last_active":"2024-04-25T12:05:05.681359+0000","last_peered":"2024-04-25T12:05:05.681359+0000","last_clean":"2024-04-25T12:05:05.681359+0000","last_became_active":"2024-04-25T12:05:03.658521+0000","last_became_peered":"2024-04-25T12:05:03.658521+0000","last_unstale":"2024-04-25T12:05:05.681359+0000","last_undegraded":"2024-04-25T12:05:05.681359+0000","last_fullsized":"2024-04-25T12:05:05.681359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:21.296484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.648376+0000","last_change":"2024-04-25T12:05:04.665958+0000","last_active":"2024-04-25T12:05:05.648376+0000","last_peered":"2024-04-25T12:05:05.648376+0000","last_clean":"2024-04-25T12:05:05.648376+0000","last_became_active":"2024-04-25T12:05:04.665619+0000","last_became_peered":"2024-04-25T12:05:04.665619+0000","last_unstale":"2024-04-25T12:05:05.648376+0000","last_undegraded":"2024-04-25T12:05:05.648376+0000","last_fullsized":"2024-04-25T12:05:05.648376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:46.837053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014276+0000","last_change":"2024-04-25T12:05:03.661021+0000","last_active":"2024-04-25T12:05:06.014276+0000","last_peered":"2024-04-25T12:05:06.014276+0000","last_clean":"2024-04-25T12:05:06.014276+0000","last_became_active":"2024-04-25T12:05:03.660690+0000","last_became_peered":"2024-04-25T12:05:03.660690+0000","last_unstale":"2024-04-25T12:05:06.014276+0000","last_undegraded":"2024-04-25T12:05:06.014276+0000","last_fullsized":"2024-04-25T12:05:06.014276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:00.645287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.648224+0000","last_change":"2024-04-25T12:05:04.669496+0000","last_active":"2024-04-25T12:05:05.648224+0000","last_peered":"2024-04-25T12:05:05.648224+0000","last_clean":"2024-04-25T12:05:05.648224+0000","last_became_active":"2024-04-25T12:05:04.669173+0000","last_became_peered":"2024-04-25T12:05:04.669173+0000","last_unstale":"2024-04-25T12:05:05.648224+0000","last_undegraded":"2024-04-25T12:05:05.648224+0000","last_fullsized":"2024-04-25T12:05:05.648224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:30.527192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756547+0000","last_change":"2024-04-25T12:05:04.673622+0000","last_active":"2024-04-25T12:05:06.756547+0000","last_peered":"2024-04-25T12:05:06.756547+0000","last_clean":"2024-04-25T12:05:06.756547+0000","last_became_active":"2024-04-25T12:05:04.673245+0000","last_became_peered":"2024-04-25T12:05:04.673245+0000","last_unstale":"2024-04-25T12:05:06.756547+0000","last_undegraded":"2024-04-25T12:05:06.756547+0000","last_fullsized":"2024-04-25T12:05:06.756547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:59.180387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.646887+0000","last_change":"2024-04-25T12:05:03.665419+0000","last_active":"2024-04-25T12:05:05.646887+0000","last_peered":"2024-04-25T12:05:05.646887+0000","last_clean":"2024-04-25T12:05:05.646887+0000","last_became_active":"2024-04-25T12:05:03.664963+0000","last_became_peered":"2024-04-25T12:05:03.664963+0000","last_unstale":"2024-04-25T12:05:05.646887+0000","last_undegraded":"2024-04-25T12:05:05.646887+0000","last_fullsized":"2024-04-25T12:05:05.646887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:04.074104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683452+0000","last_change":"2024-04-25T12:05:04.674257+0000","last_active":"2024-04-25T12:05:05.683452+0000","last_peered":"2024-04-25T12:05:05.683452+0000","last_clean":"2024-04-25T12:05:05.683452+0000","last_became_active":"2024-04-25T12:05:04.669726+0000","last_became_peered":"2024-04-25T12:05:04.669726+0000","last_unstale":"2024-04-25T12:05:05.683452+0000","last_undegraded":"2024-04-25T12:05:05.683452+0000","last_fullsized":"2024-04-25T12:05:05.683452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:03.304088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756407+0000","last_change":"2024-04-25T12:05:03.652149+0000","last_active":"2024-04-25T12:05:06.756407+0000","last_peered":"2024-04-25T12:05:06.756407+0000","last_clean":"2024-04-25T12:05:06.756407+0000","last_became_active":"2024-04-25T12:05:03.650216+0000","last_became_peered":"2024-04-25T12:05:03.650216+0000","last_unstale":"2024-04-25T12:05:06.756407+0000","last_undegraded":"2024-04-25T12:05:06.756407+0000","last_fullsized":"2024-04-25T12:05:06.756407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:00.255226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014091+0000","last_change":"2024-04-25T12:03:51.170356+0000","last_active":"2024-04-25T12:05:06.014091+0000","last_peered":"2024-04-25T12:05:06.014091+0000","last_clean":"2024-04-25T12:05:06.014091+0000","last_became_active":"2024-04-25T12:03:51.169795+0000","last_became_peered":"2024-04-25T12:03:51.169795+0000","last_unstale":"2024-04-25T12:05:06.014091+0000","last_undegraded":"2024-04-25T12:05:06.014091+0000","last_fullsized":"2024-04-25T12:05:06.014091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:00:21.810858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:00:21.810858+0000","last_clean_scrub_stamp":"2024-04-25T12:00:21.810858+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:34.246289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.646943+0000","last_change":"2024-04-25T12:05:04.671882+0000","last_active":"2024-04-25T12:05:05.646943+0000","last_peered":"2024-04-25T12:05:05.646943+0000","last_clean":"2024-04-25T12:05:05.646943+0000","last_became_active":"2024-04-25T12:05:04.671408+0000","last_became_peered":"2024-04-25T12:05:04.671408+0000","last_unstale":"2024-04-25T12:05:05.646943+0000","last_undegraded":"2024-04-25T12:05:05.646943+0000","last_fullsized":"2024-04-25T12:05:05.646943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:36.465054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:08.708537+0000","last_change":"2024-04-25T12:05:03.659082+0000","last_active":"2024-04-25T12:05:08.708537+0000","last_peered":"2024-04-25T12:05:08.708537+0000","last_clean":"2024-04-25T12:05:08.708537+0000","last_became_active":"2024-04-25T12:05:03.658677+0000","last_became_peered":"2024-04-25T12:05:03.658677+0000","last_unstale":"2024-04-25T12:05:08.708537+0000","last_undegraded":"2024-04-25T12:05:08.708537+0000","last_fullsized":"2024-04-25T12:05:08.708537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:20.938394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:09.721821+0000","last_change":"2024-04-25T12:05:03.661172+0000","last_active":"2024-04-25T12:05:09.721821+0000","last_peered":"2024-04-25T12:05:09.721821+0000","last_clean":"2024-04-25T12:05:09.721821+0000","last_became_active":"2024-04-25T12:05:03.660799+0000","last_became_peered":"2024-04-25T12:05:03.660799+0000","last_unstale":"2024-04-25T12:05:09.721821+0000","last_undegraded":"2024-04-25T12:05:09.721821+0000","last_fullsized":"2024-04-25T12:05:09.721821+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:01.889714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.647180+0000","last_change":"2024-04-25T12:05:04.662203+0000","last_active":"2024-04-25T12:05:05.647180+0000","last_peered":"2024-04-25T12:05:05.647180+0000","last_clean":"2024-04-25T12:05:05.647180+0000","last_became_active":"2024-04-25T12:05:04.661816+0000","last_became_peered":"2024-04-25T12:05:04.661816+0000","last_unstale":"2024-04-25T12:05:05.647180+0000","last_undegraded":"2024-04-25T12:05:05.647180+0000","last_fullsized":"2024-04-25T12:05:05.647180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:06.656668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.682955+0000","last_change":"2024-04-25T12:05:03.664741+0000","last_active":"2024-04-25T12:05:05.682955+0000","last_peered":"2024-04-25T12:05:05.682955+0000","last_clean":"2024-04-25T12:05:05.682955+0000","last_became_active":"2024-04-25T12:05:03.664373+0000","last_became_peered":"2024-04-25T12:05:03.664373+0000","last_unstale":"2024-04-25T12:05:05.682955+0000","last_undegraded":"2024-04-25T12:05:05.682955+0000","last_fullsized":"2024-04-25T12:05:05.682955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:03.540606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683675+0000","last_change":"2024-04-25T12:05:04.677748+0000","last_active":"2024-04-25T12:05:05.683675+0000","last_peered":"2024-04-25T12:05:05.683675+0000","last_clean":"2024-04-25T12:05:05.683675+0000","last_became_active":"2024-04-25T12:05:04.674003+0000","last_became_peered":"2024-04-25T12:05:04.674003+0000","last_unstale":"2024-04-25T12:05:05.683675+0000","last_undegraded":"2024-04-25T12:05:05.683675+0000","last_fullsized":"2024-04-25T12:05:05.683675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:24.031333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756375+0000","last_change":"2024-04-25T12:05:03.666295+0000","last_active":"2024-04-25T12:05:06.756375+0000","last_peered":"2024-04-25T12:05:06.756375+0000","last_clean":"2024-04-25T12:05:06.756375+0000","last_became_active":"2024-04-25T12:05:03.665910+0000","last_became_peered":"2024-04-25T12:05:03.665910+0000","last_unstale":"2024-04-25T12:05:06.756375+0000","last_undegraded":"2024-04-25T12:05:06.756375+0000","last_fullsized":"2024-04-25T12:05:06.756375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:13.028799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756503+0000","last_change":"2024-04-25T12:05:04.668765+0000","last_active":"2024-04-25T12:05:06.756503+0000","last_peered":"2024-04-25T12:05:06.756503+0000","last_clean":"2024-04-25T12:05:06.756503+0000","last_became_active":"2024-04-25T12:05:04.668391+0000","last_became_peered":"2024-04-25T12:05:04.668391+0000","last_unstale":"2024-04-25T12:05:06.756503+0000","last_undegraded":"2024-04-25T12:05:06.756503+0000","last_fullsized":"2024-04-25T12:05:06.756503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:47.282921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:07.703946+0000","last_change":"2024-04-25T12:05:03.663799+0000","last_active":"2024-04-25T12:05:07.703946+0000","last_peered":"2024-04-25T12:05:07.703946+0000","last_clean":"2024-04-25T12:05:07.703946+0000","last_became_active":"2024-04-25T12:05:03.663501+0000","last_became_peered":"2024-04-25T12:05:03.663501+0000","last_unstale":"2024-04-25T12:05:07.703946+0000","last_undegraded":"2024-04-25T12:05:07.703946+0000","last_fullsized":"2024-04-25T12:05:07.703946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:34.402036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683179+0000","last_change":"2024-04-25T12:05:04.662594+0000","last_active":"2024-04-25T12:05:05.683179+0000","last_peered":"2024-04-25T12:05:05.683179+0000","last_clean":"2024-04-25T12:05:05.683179+0000","last_became_active":"2024-04-25T12:05:04.662179+0000","last_became_peered":"2024-04-25T12:05:04.662179+0000","last_unstale":"2024-04-25T12:05:05.683179+0000","last_undegraded":"2024-04-25T12:05:05.683179+0000","last_fullsized":"2024-04-25T12:05:05.683179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:58.709837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683135+0000","last_change":"2024-04-25T12:05:03.659416+0000","last_active":"2024-04-25T12:05:05.683135+0000","last_peered":"2024-04-25T12:05:05.683135+0000","last_clean":"2024-04-25T12:05:05.683135+0000","last_became_active":"2024-04-25T12:05:03.658416+0000","last_became_peered":"2024-04-25T12:05:03.658416+0000","last_unstale":"2024-04-25T12:05:05.683135+0000","last_undegraded":"2024-04-25T12:05:05.683135+0000","last_fullsized":"2024-04-25T12:05:05.683135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:01.174438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.654872+0000","last_change":"2024-04-25T12:05:04.670890+0000","last_active":"2024-04-25T12:05:05.654872+0000","last_peered":"2024-04-25T12:05:05.654872+0000","last_clean":"2024-04-25T12:05:05.654872+0000","last_became_active":"2024-04-25T12:05:04.670581+0000","last_became_peered":"2024-04-25T12:05:04.670581+0000","last_unstale":"2024-04-25T12:05:05.654872+0000","last_undegraded":"2024-04-25T12:05:05.654872+0000","last_fullsized":"2024-04-25T12:05:05.654872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.527303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:09.720046+0000","last_change":"2024-04-25T12:05:03.662561+0000","last_active":"2024-04-25T12:05:09.720046+0000","last_peered":"2024-04-25T12:05:09.720046+0000","last_clean":"2024-04-25T12:05:09.720046+0000","last_became_active":"2024-04-25T12:05:03.662155+0000","last_became_peered":"2024-04-25T12:05:03.662155+0000","last_unstale":"2024-04-25T12:05:09.720046+0000","last_undegraded":"2024-04-25T12:05:09.720046+0000","last_fullsized":"2024-04-25T12:05:09.720046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:02.130271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683492+0000","last_change":"2024-04-25T12:05:04.669905+0000","last_active":"2024-04-25T12:05:05.683492+0000","last_peered":"2024-04-25T12:05:05.683492+0000","last_clean":"2024-04-25T12:05:05.683492+0000","last_became_active":"2024-04-25T12:05:04.669322+0000","last_became_peered":"2024-04-25T12:05:04.669322+0000","last_unstale":"2024-04-25T12:05:05.683492+0000","last_undegraded":"2024-04-25T12:05:05.683492+0000","last_fullsized":"2024-04-25T12:05:05.683492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.705643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:07.704599+0000","last_change":"2024-04-25T12:05:03.657923+0000","last_active":"2024-04-25T12:05:07.704599+0000","last_peered":"2024-04-25T12:05:07.704599+0000","last_clean":"2024-04-25T12:05:07.704599+0000","last_became_active":"2024-04-25T12:05:03.657332+0000","last_became_peered":"2024-04-25T12:05:03.657332+0000","last_unstale":"2024-04-25T12:05:07.704599+0000","last_undegraded":"2024-04-25T12:05:07.704599+0000","last_fullsized":"2024-04-25T12:05:07.704599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:54.521822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014393+0000","last_change":"2024-04-25T12:05:04.670193+0000","last_active":"2024-04-25T12:05:06.014393+0000","last_peered":"2024-04-25T12:05:06.014393+0000","last_clean":"2024-04-25T12:05:06.014393+0000","last_became_active":"2024-04-25T12:05:04.669720+0000","last_became_peered":"2024-04-25T12:05:04.669720+0000","last_unstale":"2024-04-25T12:05:06.014393+0000","last_undegraded":"2024-04-25T12:05:06.014393+0000","last_fullsized":"2024-04-25T12:05:06.014393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:26.080914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:10.184866+0000","last_change":"2024-04-25T12:05:03.666064+0000","last_active":"2024-04-25T12:05:10.184866+0000","last_peered":"2024-04-25T12:05:10.184866+0000","last_clean":"2024-04-25T12:05:10.184866+0000","last_became_active":"2024-04-25T12:05:03.665749+0000","last_became_peered":"2024-04-25T12:05:03.665749+0000","last_unstale":"2024-04-25T12:05:10.184866+0000","last_undegraded":"2024-04-25T12:05:10.184866+0000","last_fullsized":"2024-04-25T12:05:10.184866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:21.607549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014619+0000","last_change":"2024-04-25T12:05:04.660953+0000","last_active":"2024-04-25T12:05:06.014619+0000","last_peered":"2024-04-25T12:05:06.014619+0000","last_clean":"2024-04-25T12:05:06.014619+0000","last_became_active":"2024-04-25T12:05:04.660577+0000","last_became_peered":"2024-04-25T12:05:04.660577+0000","last_unstale":"2024-04-25T12:05:06.014619+0000","last_undegraded":"2024-04-25T12:05:06.014619+0000","last_fullsized":"2024-04-25T12:05:06.014619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:35.728839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756173+0000","last_change":"2024-04-25T12:05:03.664454+0000","last_active":"2024-04-25T12:05:06.756173+0000","last_peered":"2024-04-25T12:05:06.756173+0000","last_clean":"2024-04-25T12:05:06.756173+0000","last_became_active":"2024-04-25T12:05:03.664064+0000","last_became_peered":"2024-04-25T12:05:03.664064+0000","last_unstale":"2024-04-25T12:05:06.756173+0000","last_undegraded":"2024-04-25T12:05:06.756173+0000","last_fullsized":"2024-04-25T12:05:06.756173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:28.146713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014403+0000","last_change":"2024-04-25T12:05:04.668385+0000","last_active":"2024-04-25T12:05:06.014403+0000","last_peered":"2024-04-25T12:05:06.014403+0000","last_clean":"2024-04-25T12:05:06.014403+0000","last_became_active":"2024-04-25T12:05:04.667995+0000","last_became_peered":"2024-04-25T12:05:04.667995+0000","last_unstale":"2024-04-25T12:05:06.014403+0000","last_undegraded":"2024-04-25T12:05:06.014403+0000","last_fullsized":"2024-04-25T12:05:06.014403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:39.938151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.647278+0000","last_change":"2024-04-25T12:05:03.661795+0000","last_active":"2024-04-25T12:05:05.647278+0000","last_peered":"2024-04-25T12:05:05.647278+0000","last_clean":"2024-04-25T12:05:05.647278+0000","last_became_active":"2024-04-25T12:05:03.661443+0000","last_became_peered":"2024-04-25T12:05:03.661443+0000","last_unstale":"2024-04-25T12:05:05.647278+0000","last_undegraded":"2024-04-25T12:05:05.647278+0000","last_fullsized":"2024-04-25T12:05:05.647278+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:52:52.901723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.014682+0000","last_change":"2024-04-25T12:05:04.662068+0000","last_active":"2024-04-25T12:05:06.014682+0000","last_peered":"2024-04-25T12:05:06.014682+0000","last_clean":"2024-04-25T12:05:06.014682+0000","last_became_active":"2024-04-25T12:05:04.661694+0000","last_became_peered":"2024-04-25T12:05:04.661694+0000","last_unstale":"2024-04-25T12:05:06.014682+0000","last_undegraded":"2024-04-25T12:05:06.014682+0000","last_fullsized":"2024-04-25T12:05:06.014682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:45.200872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.655010+0000","last_change":"2024-04-25T12:05:04.662320+0000","last_active":"2024-04-25T12:05:05.655010+0000","last_peered":"2024-04-25T12:05:05.655010+0000","last_clean":"2024-04-25T12:05:05.655010+0000","last_became_active":"2024-04-25T12:05:04.661992+0000","last_became_peered":"2024-04-25T12:05:04.661992+0000","last_unstale":"2024-04-25T12:05:05.655010+0000","last_undegraded":"2024-04-25T12:05:05.655010+0000","last_fullsized":"2024-04-25T12:05:05.655010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:25.898506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.650641+0000","last_change":"2024-04-25T12:05:04.671534+0000","last_active":"2024-04-25T12:05:05.650641+0000","last_peered":"2024-04-25T12:05:05.650641+0000","last_clean":"2024-04-25T12:05:05.650641+0000","last_became_active":"2024-04-25T12:05:04.671169+0000","last_became_peered":"2024-04-25T12:05:04.671169+0000","last_unstale":"2024-04-25T12:05:05.650641+0000","last_undegraded":"2024-04-25T12:05:05.650641+0000","last_fullsized":"2024-04-25T12:05:05.650641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:49.972884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.650680+0000","last_change":"2024-04-25T12:05:04.668832+0000","last_active":"2024-04-25T12:05:05.650680+0000","last_peered":"2024-04-25T12:05:05.650680+0000","last_clean":"2024-04-25T12:05:05.650680+0000","last_became_active":"2024-04-25T12:05:04.668459+0000","last_became_peered":"2024-04-25T12:05:04.668459+0000","last_unstale":"2024-04-25T12:05:05.650680+0000","last_undegraded":"2024-04-25T12:05:05.650680+0000","last_fullsized":"2024-04-25T12:05:05.650680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:58.509505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756336+0000","last_change":"2024-04-25T12:05:04.671393+0000","last_active":"2024-04-25T12:05:06.756336+0000","last_peered":"2024-04-25T12:05:06.756336+0000","last_clean":"2024-04-25T12:05:06.756336+0000","last_became_active":"2024-04-25T12:05:04.671041+0000","last_became_peered":"2024-04-25T12:05:04.671041+0000","last_unstale":"2024-04-25T12:05:06.756336+0000","last_undegraded":"2024-04-25T12:05:06.756336+0000","last_fullsized":"2024-04-25T12:05:06.756336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:29.428573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.647368+0000","last_change":"2024-04-25T12:05:04.669936+0000","last_active":"2024-04-25T12:05:05.647368+0000","last_peered":"2024-04-25T12:05:05.647368+0000","last_clean":"2024-04-25T12:05:05.647368+0000","last_became_active":"2024-04-25T12:05:04.669204+0000","last_became_peered":"2024-04-25T12:05:04.669204+0000","last_unstale":"2024-04-25T12:05:05.647368+0000","last_undegraded":"2024-04-25T12:05:05.647368+0000","last_fullsized":"2024-04-25T12:05:05.647368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:38.814562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.655239+0000","last_change":"2024-04-25T12:05:04.664852+0000","last_active":"2024-04-25T12:05:05.655239+0000","last_peered":"2024-04-25T12:05:05.655239+0000","last_clean":"2024-04-25T12:05:05.655239+0000","last_became_active":"2024-04-25T12:05:04.663269+0000","last_became_peered":"2024-04-25T12:05:04.663269+0000","last_unstale":"2024-04-25T12:05:05.655239+0000","last_undegraded":"2024-04-25T12:05:05.655239+0000","last_fullsized":"2024-04-25T12:05:05.655239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:57.862378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.651005+0000","last_change":"2024-04-25T12:05:04.666332+0000","last_active":"2024-04-25T12:05:05.651005+0000","last_peered":"2024-04-25T12:05:05.651005+0000","last_clean":"2024-04-25T12:05:05.651005+0000","last_became_active":"2024-04-25T12:05:04.665777+0000","last_became_peered":"2024-04-25T12:05:04.665777+0000","last_unstale":"2024-04-25T12:05:05.651005+0000","last_undegraded":"2024-04-25T12:05:05.651005+0000","last_fullsized":"2024-04-25T12:05:05.651005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:53.912782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683581+0000","last_change":"2024-04-25T12:05:04.678238+0000","last_active":"2024-04-25T12:05:05.683581+0000","last_peered":"2024-04-25T12:05:05.683581+0000","last_clean":"2024-04-25T12:05:05.683581+0000","last_became_active":"2024-04-25T12:05:04.677660+0000","last_became_peered":"2024-04-25T12:05:04.677660+0000","last_unstale":"2024-04-25T12:05:05.683581+0000","last_undegraded":"2024-04-25T12:05:05.683581+0000","last_fullsized":"2024-04-25T12:05:05.683581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:02.916486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.756213+0000","last_change":"2024-04-25T12:05:04.666533+0000","last_active":"2024-04-25T12:05:06.756213+0000","last_peered":"2024-04-25T12:05:06.756213+0000","last_clean":"2024-04-25T12:05:06.756213+0000","last_became_active":"2024-04-25T12:05:04.666184+0000","last_became_peered":"2024-04-25T12:05:04.666184+0000","last_unstale":"2024-04-25T12:05:06.756213+0000","last_undegraded":"2024-04-25T12:05:06.756213+0000","last_fullsized":"2024-04-25T12:05:06.756213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:13.089285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.683272+0000","last_change":"2024-04-25T12:05:04.672880+0000","last_active":"2024-04-25T12:05:05.683272+0000","last_peered":"2024-04-25T12:05:05.683272+0000","last_clean":"2024-04-25T12:05:05.683272+0000","last_became_active":"2024-04-25T12:05:04.669640+0000","last_became_peered":"2024-04-25T12:05:04.669640+0000","last_unstale":"2024-04-25T12:05:05.683272+0000","last_undegraded":"2024-04-25T12:05:05.683272+0000","last_fullsized":"2024-04-25T12:05:05.683272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:52.293914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:06.636078+0000","last_change":"2024-04-25T12:05:04.664274+0000","last_active":"2024-04-25T12:05:06.636078+0000","last_peered":"2024-04-25T12:05:06.636078+0000","last_clean":"2024-04-25T12:05:06.636078+0000","last_became_active":"2024-04-25T12:05:04.663715+0000","last_became_peered":"2024-04-25T12:05:04.663715+0000","last_unstale":"2024-04-25T12:05:06.636078+0000","last_undegraded":"2024-04-25T12:05:06.636078+0000","last_fullsized":"2024-04-25T12:05:06.636078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:28.298147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.647178+0000","last_change":"2024-04-25T12:05:04.669974+0000","last_active":"2024-04-25T12:05:05.647178+0000","last_peered":"2024-04-25T12:05:05.647178+0000","last_clean":"2024-04-25T12:05:05.647178+0000","last_became_active":"2024-04-25T12:05:04.669531+0000","last_became_peered":"2024-04-25T12:05:04.669531+0000","last_unstale":"2024-04-25T12:05:05.647178+0000","last_undegraded":"2024-04-25T12:05:05.647178+0000","last_fullsized":"2024-04-25T12:05:05.647178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:51.579582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T12:05:05.655038+0000","last_change":"2024-04-25T12:05:04.670203+0000","last_active":"2024-04-25T12:05:05.655038+0000","last_peered":"2024-04-25T12:05:05.655038+0000","last_clean":"2024-04-25T12:05:05.655038+0000","last_became_active":"2024-04-25T12:05:04.669865+0000","last_became_peered":"2024-04-25T12:05:04.669865+0000","last_unstale":"2024-04-25T12:05:05.655038+0000","last_undegraded":"2024-04-25T12:05:05.655038+0000","last_fullsized":"2024-04-25T12:05:05.655038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:19.860002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281216,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4952,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":98,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":33557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645330,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":1030695,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070494,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":1030681,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":439215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":461388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":909312,"data_stored":430055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":438543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":1030365,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34127,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34127,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34127,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:05:12.898 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T12:05:12.898 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T12:05:12.898 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T12:05:13.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:12 smithi169 ceph-mon[32710]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:13.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:12 smithi169 ceph-mon[32710]: pgmap v293: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 564 B/s rd, 4.0 KiB/s wr, 14 op/s 2024-04-25T12:05:13.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:12 smithi169 ceph-mon[32710]: from='client.24625 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:13.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:12 smithi169 ceph-mon[32710]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:13.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:12 smithi169 ceph-mon[32710]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T12:05:13.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:12 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2290166817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:05:13.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:12 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1451737908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:13.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:12 smithi150 ceph-mon[32250]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:13.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:12 smithi150 ceph-mon[32250]: pgmap v293: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 564 B/s rd, 4.0 KiB/s wr, 14 op/s 2024-04-25T12:05:13.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:12 smithi150 ceph-mon[32250]: from='client.24625 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:13.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:12 smithi150 ceph-mon[32250]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:05:13.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:12 smithi150 ceph-mon[32250]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T12:05:13.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:12 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2290166817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:05:13.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:12 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1451737908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:13.179 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:14.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:13 smithi019 ceph-mon[29213]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:05:14.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:13 smithi169 ceph-mon[32710]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:05:14.162 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:13 smithi150 ceph-mon[32250]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:05:14.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:14 smithi169 ceph-mon[32710]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:05:14.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:14 smithi169 ceph-mon[32710]: pgmap v294: 97 pgs: 97 active+clean; 619 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-04-25T12:05:14.906 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:05:14 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i[64470]: 2024-04-25T12:05:14.557+0000 7faf51719700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-25T12:05:14.906 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:05:14 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i[64470]: 2024-04-25T12:05:14.557+0000 7faf51719700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-25T12:05:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.962+0000 7faec6b4f700 1 -- 172.21.15.19:0/3700417532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faec0102be0 msgr2=0x7faec0103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.962+0000 7faec6b4f700 1 --2- 172.21.15.19:0/3700417532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faec0102be0 0x7faec0103040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faeb40099c0 tx=0x7faeb4009cd0 comp rx=0 tx=0).stop 2024-04-25T12:05:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.962+0000 7faec6b4f700 1 -- 172.21.15.19:0/3700417532 shutdown_connections 2024-04-25T12:05:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.962+0000 7faec6b4f700 1 --2- 172.21.15.19:0/3700417532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faec0103580 0x7faec0109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.962+0000 7faec6b4f700 1 --2- 172.21.15.19:0/3700417532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faec01019e0 0x7faec0101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.962+0000 7faec6b4f700 1 --2- 172.21.15.19:0/3700417532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faec0102be0 0x7faec0103040 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:14.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.962+0000 7faec6b4f700 1 -- 172.21.15.19:0/3700417532 >> 172.21.15.19:0/3700417532 conn(0x7faec00fcfa0 msgr2=0x7faec00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:14.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:14 smithi019 ceph-mon[29213]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:05:14.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:14 smithi019 ceph-mon[29213]: pgmap v294: 97 pgs: 97 active+clean; 619 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-04-25T12:05:14.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.962+0000 7faec6b4f700 1 -- 172.21.15.19:0/3700417532 shutdown_connections 2024-04-25T12:05:14.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.962+0000 7faec6b4f700 1 -- 172.21.15.19:0/3700417532 wait complete. 2024-04-25T12:05:14.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec6b4f700 1 Processor -- start 2024-04-25T12:05:14.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec6b4f700 1 -- start start 2024-04-25T12:05:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec6b4f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faec01019e0 0x7faec0198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec6b4f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faec0102be0 0x7faec01990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec6b4f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faec0103580 0x7faec01a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec6b4f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faec010d5e0 con 0x7faec0103580 2024-04-25T12:05:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec6b4f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faec010d460 con 0x7faec01019e0 2024-04-25T12:05:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec6b4f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7faec010d760 con 0x7faec0102be0 2024-04-25T12:05:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec50ec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faec0103580 0x7faec01a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec50ec700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faec0103580 0x7faec01a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42746/0 (socket says 172.21.15.19:42746) 2024-04-25T12:05:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec50ec700 1 -- 172.21.15.19:0/2553207413 learned_addr learned my addr 172.21.15.19:0/2553207413 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faebffff700 1 --2- 172.21.15.19:0/2553207413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faec0102be0 0x7faec01990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec48eb700 1 --2- 172.21.15.19:0/2553207413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faec01019e0 0x7faec0198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec50ec700 1 -- 172.21.15.19:0/2553207413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faec0102be0 msgr2=0x7faec01990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec50ec700 1 --2- 172.21.15.19:0/2553207413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faec0102be0 0x7faec01990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:14.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec50ec700 1 -- 172.21.15.19:0/2553207413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faec01019e0 msgr2=0x7faec0198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec50ec700 1 --2- 172.21.15.19:0/2553207413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faec01019e0 0x7faec0198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.963+0000 7faec50ec700 1 -- 172.21.15.19:0/2553207413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec01a0800 con 0x7faec0103580 2024-04-25T12:05:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.964+0000 7faec50ec700 1 --2- 172.21.15.19:0/2553207413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faec0103580 0x7faec01a0160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faeb800add0 tx=0x7faeb800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.964+0000 7faebdffb700 1 -- 172.21.15.19:0/2553207413 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb8017020 con 0x7faec0103580 2024-04-25T12:05:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.964+0000 7faebdffb700 1 -- 172.21.15.19:0/2553207413 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7faeb800ce10 con 0x7faec0103580 2024-04-25T12:05:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.964+0000 7faebdffb700 1 -- 172.21.15.19:0/2553207413 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb80053a0 con 0x7faec0103580 2024-04-25T12:05:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.964+0000 7faec6b4f700 1 -- 172.21.15.19:0/2553207413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec01a0af0 con 0x7faec0103580 2024-04-25T12:05:14.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.964+0000 7faec6b4f700 1 -- 172.21.15.19:0/2553207413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec01a1030 con 0x7faec0103580 2024-04-25T12:05:14.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.965+0000 7faec6b4f700 1 -- 172.21.15.19:0/2553207413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec0050f80 con 0x7faec0103580 2024-04-25T12:05:14.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.966+0000 7faebdffb700 1 -- 172.21.15.19:0/2553207413 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faeb800c940 con 0x7faec0103580 2024-04-25T12:05:14.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.967+0000 7faebdffb700 1 --2- 172.21.15.19:0/2553207413 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7faea80a3d80 0x7faea80a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:14.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.967+0000 7faec48eb700 1 --2- 172.21.15.19:0/2553207413 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7faea80a3d80 0x7faea80a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:14.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.967+0000 7faebdffb700 1 -- 172.21.15.19:0/2553207413 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7faeb80cfb30 con 0x7faec0103580 2024-04-25T12:05:14.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.968+0000 7faec48eb700 1 --2- 172.21.15.19:0/2553207413 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7faea80a3d80 0x7faea80a6240 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faeb0005fd0 tx=0x7faeb0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:14.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:14.972+0000 7faebdffb700 1 -- 172.21.15.19:0/2553207413 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faeb8021020 con 0x7faec0103580 2024-04-25T12:05:15.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:14 smithi150 ceph-mon[32250]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:05:15.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:14 smithi150 ceph-mon[32250]: pgmap v294: 97 pgs: 97 active+clean; 619 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-04-25T12:05:15.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:15.265+0000 7faec6b4f700 1 -- 172.21.15.19:0/2553207413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7faec01a1460 con 0x7faec0103580 2024-04-25T12:05:15.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:15.270+0000 7faebdffb700 1 -- 172.21.15.19:0/2553207413 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7faec01a1460 con 0x7faec0103580 2024-04-25T12:05:15.267 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-04-25T12:05:15.267 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQD7Ripmix3jDxAAxD9iys/OmwheQL9CVukJKQ== 2024-04-25T12:05:15.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:15.273+0000 7faec6b4f700 1 -- 172.21.15.19:0/2553207413 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7faea80a3d80 msgr2=0x7faea80a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:15.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:15.273+0000 7faec6b4f700 1 --2- 172.21.15.19:0/2553207413 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7faea80a3d80 0x7faea80a6240 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faeb0005fd0 tx=0x7faeb0005e50 comp rx=0 tx=0).stop 2024-04-25T12:05:15.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:15.273+0000 7faec6b4f700 1 -- 172.21.15.19:0/2553207413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faec0103580 msgr2=0x7faec01a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:15.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:15.273+0000 7faec6b4f700 1 --2- 172.21.15.19:0/2553207413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faec0103580 0x7faec01a0160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faeb800add0 tx=0x7faeb800c330 comp rx=0 tx=0).stop 2024-04-25T12:05:15.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:15.273+0000 7faec6b4f700 1 -- 172.21.15.19:0/2553207413 shutdown_connections 2024-04-25T12:05:15.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:15.273+0000 7faec6b4f700 1 --2- 172.21.15.19:0/2553207413 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7faea80a3d80 0x7faea80a6240 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:15.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:15.273+0000 7faec6b4f700 1 --2- 172.21.15.19:0/2553207413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faec0103580 0x7faec01a0160 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:15.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:15.273+0000 7faec6b4f700 1 --2- 172.21.15.19:0/2553207413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faec01019e0 0x7faec0198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:15.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:15.273+0000 7faec6b4f700 1 --2- 172.21.15.19:0/2553207413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faec0102be0 0x7faec01990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:15.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:15.273+0000 7faec6b4f700 1 -- 172.21.15.19:0/2553207413 >> 172.21.15.19:0/2553207413 conn(0x7faec00fcfa0 msgr2=0x7faec006b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:15.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:15.273+0000 7faec6b4f700 1 -- 172.21.15.19:0/2553207413 shutdown_connections 2024-04-25T12:05:15.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:15.273+0000 7faec6b4f700 1 -- 172.21.15.19:0/2553207413 wait complete. 2024-04-25T12:05:15.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:15 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2553207413' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T12:05:15.904 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:15 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2553207413' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T12:05:15.934 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T12:05:15.934 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T12:05:15.934 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T12:05:15.973 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T12:05:15.973 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T12:05:15.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T12:05:16.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:15 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2553207413' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T12:05:16.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:15 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2553207413' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T12:05:16.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:15 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2553207413' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T12:05:16.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:15 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2553207413' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T12:05:16.262 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:17.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:16 smithi019 ceph-mon[29213]: pgmap v295: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-04-25T12:05:17.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:16 smithi169 ceph-mon[32710]: pgmap v295: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-04-25T12:05:17.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:16 smithi150 ceph-mon[32250]: pgmap v295: 97 pgs: 97 active+clean; 620 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-04-25T12:05:18.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.000+0000 7f1db86a5700 1 -- 172.21.15.19:0/790132905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1db0108ab0 msgr2=0x7f1db0108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:18.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.000+0000 7f1db86a5700 1 --2- 172.21.15.19:0/790132905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1db0108ab0 0x7f1db0108e90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1da000b0a0 tx=0x7f1da000b3b0 comp rx=0 tx=0).stop 2024-04-25T12:05:18.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.001+0000 7f1db86a5700 1 -- 172.21.15.19:0/790132905 shutdown_connections 2024-04-25T12:05:18.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.001+0000 7f1db86a5700 1 --2- 172.21.15.19:0/790132905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1db0108ab0 0x7f1db0108e90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:18.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.001+0000 7f1db86a5700 1 --2- 172.21.15.19:0/790132905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1db0103480 0x7f1db010f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:18.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.001+0000 7f1db86a5700 1 --2- 172.21.15.19:0/790132905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db0102ac0 0x7f1db0102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:18.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.001+0000 7f1db86a5700 1 -- 172.21.15.19:0/790132905 >> 172.21.15.19:0/790132905 conn(0x7f1db0079bf0 msgr2=0x7f1db007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:18.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.001+0000 7f1db86a5700 1 -- 172.21.15.19:0/790132905 shutdown_connections 2024-04-25T12:05:18.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.001+0000 7f1db86a5700 1 -- 172.21.15.19:0/790132905 wait complete. 2024-04-25T12:05:18.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.001+0000 7f1db86a5700 1 Processor -- start 2024-04-25T12:05:18.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.001+0000 7f1db86a5700 1 -- start start 2024-04-25T12:05:18.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db86a5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1db0102ac0 0x7f1db01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:18.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db86a5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1db0103480 0x7f1db01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:18.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db86a5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db0108ab0 0x7f1db019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:18.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db86a5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1db0112e10 con 0x7f1db0102ac0 2024-04-25T12:05:18.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db86a5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1db0112c90 con 0x7f1db0103480 2024-04-25T12:05:18.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db86a5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1db0112f90 con 0x7f1db0108ab0 2024-04-25T12:05:18.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db6441700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1db0102ac0 0x7f1db01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:18.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db6441700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1db0102ac0 0x7f1db01992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42768/0 (socket says 172.21.15.19:42768) 2024-04-25T12:05:18.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db6441700 1 -- 172.21.15.19:0/1558623547 learned_addr learned my addr 172.21.15.19:0/1558623547 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:18.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db5c40700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1db0103480 0x7f1db01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:18.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db6c42700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db0108ab0 0x7f1db019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:18.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db6441700 1 -- 172.21.15.19:0/1558623547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db0108ab0 msgr2=0x7f1db019db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:18.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db6441700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db0108ab0 0x7f1db019db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:18.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db6441700 1 -- 172.21.15.19:0/1558623547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1db0103480 msgr2=0x7f1db01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:18.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db6441700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1db0103480 0x7f1db01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:18.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db6441700 1 -- 172.21.15.19:0/1558623547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db019e260 con 0x7f1db0102ac0 2024-04-25T12:05:18.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db6c42700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db0108ab0 0x7f1db019db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:05:18.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.002+0000 7f1db6441700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1db0102ac0 0x7f1db01992a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1da000b070 tx=0x7f1da0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.003+0000 7f1daf7fe700 1 -- 172.21.15.19:0/1558623547 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0021070 con 0x7f1db0102ac0 2024-04-25T12:05:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.003+0000 7f1db86a5700 1 -- 172.21.15.19:0/1558623547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db019e4f0 con 0x7f1db0102ac0 2024-04-25T12:05:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.003+0000 7f1daf7fe700 1 -- 172.21.15.19:0/1558623547 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1da0007b80 con 0x7f1db0102ac0 2024-04-25T12:05:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.003+0000 7f1daf7fe700 1 -- 172.21.15.19:0/1558623547 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da00052a0 con 0x7f1db0102ac0 2024-04-25T12:05:18.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.003+0000 7f1db86a5700 1 -- 172.21.15.19:0/1558623547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db01a5dd0 con 0x7f1db0102ac0 2024-04-25T12:05:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.004+0000 7f1db86a5700 1 -- 172.21.15.19:0/1558623547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db01041f0 con 0x7f1db0102ac0 2024-04-25T12:05:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.005+0000 7f1daf7fe700 1 -- 172.21.15.19:0/1558623547 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1da0024070 con 0x7f1db0102ac0 2024-04-25T12:05:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.006+0000 7f1daf7fe700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f1d980a3cd0 0x7f1d980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.006+0000 7f1db5c40700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f1d980a3cd0 0x7f1d980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.006+0000 7f1daf7fe700 1 -- 172.21.15.19:0/1558623547 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1da000f020 con 0x7f1db0102ac0 2024-04-25T12:05:18.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.006+0000 7f1db5c40700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f1d980a3cd0 0x7f1d980a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1db007a160 tx=0x7f1da4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:18.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.009+0000 7f1daf7fe700 1 -- 172.21.15.19:0/1558623547 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1da002b020 con 0x7f1db0102ac0 2024-04-25T12:05:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.281+0000 7f1db86a5700 1 -- 172.21.15.19:0/1558623547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f1db019a5b0 con 0x7f1db0102ac0 2024-04-25T12:05:18.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.288+0000 7f1daf7fe700 1 -- 172.21.15.19:0/1558623547 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7f1da009e7a0 con 0x7f1db0102ac0 2024-04-25T12:05:18.289 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:05:18.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.293+0000 7f1db86a5700 1 -- 172.21.15.19:0/1558623547 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f1d980a3cd0 msgr2=0x7f1d980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:18.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.293+0000 7f1db86a5700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f1d980a3cd0 0x7f1d980a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1db007a160 tx=0x7f1da4009190 comp rx=0 tx=0).stop 2024-04-25T12:05:18.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.293+0000 7f1db86a5700 1 -- 172.21.15.19:0/1558623547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1db0102ac0 msgr2=0x7f1db01992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:18.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.293+0000 7f1db86a5700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1db0102ac0 0x7f1db01992a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1da000b070 tx=0x7f1da0009810 comp rx=0 tx=0).stop 2024-04-25T12:05:18.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.293+0000 7f1db86a5700 1 -- 172.21.15.19:0/1558623547 shutdown_connections 2024-04-25T12:05:18.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.293+0000 7f1db86a5700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f1d980a3cd0 0x7f1d980a6190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:18.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.293+0000 7f1db86a5700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1db0102ac0 0x7f1db01992a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:18.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.293+0000 7f1db86a5700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1db0108ab0 0x7f1db019db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:18.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.293+0000 7f1db86a5700 1 --2- 172.21.15.19:0/1558623547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1db0103480 0x7f1db01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:18.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.293+0000 7f1db86a5700 1 -- 172.21.15.19:0/1558623547 >> 172.21.15.19:0/1558623547 conn(0x7f1db0079bf0 msgr2=0x7f1db0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:18.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.293+0000 7f1db86a5700 1 -- 172.21.15.19:0/1558623547 shutdown_connections 2024-04-25T12:05:18.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:18.293+0000 7f1db86a5700 1 -- 172.21.15.19:0/1558623547 wait complete. 2024-04-25T12:05:18.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:18 smithi019 ceph-mon[29213]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.8 KiB/s wr, 25 op/s 2024-04-25T12:05:18.941 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:18 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1558623547' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T12:05:19.015 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":2853466745},{"type":"v1","addr":"172.21.15.19:6801","nonce":2853466745}]},"active_addr":"172.21.15.19:6801/2853466745","active_change":"2024-04-25T11:57:54.812005+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.19:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1048024792}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1094943085}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":2813829603}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":2734487323}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":681589223}]}]} 2024-04-25T12:05:19.021 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T12:05:19.021 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T12:05:19.022 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T12:05:19.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:18 smithi169 ceph-mon[32710]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.8 KiB/s wr, 25 op/s 2024-04-25T12:05:19.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:18 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1558623547' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T12:05:19.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:18 smithi150 ceph-mon[32250]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.8 KiB/s wr, 25 op/s 2024-04-25T12:05:19.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:18 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1558623547' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T12:05:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:20.880 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:20 smithi019 ceph-mon[29213]: pgmap v297: 97 pgs: 97 active+clean; 626 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.4 KiB/s wr, 25 op/s 2024-04-25T12:05:21.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.128+0000 7f2e38352700 1 -- 172.21.15.19:0/2598530899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e30103480 msgr2=0x7f2e3010f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:21.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.128+0000 7f2e38352700 1 --2- 172.21.15.19:0/2598530899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e30103480 0x7f2e3010f8d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800b0a0 tx=0x7f2e2800b3b0 comp rx=0 tx=0).stop 2024-04-25T12:05:21.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.128+0000 7f2e38352700 1 -- 172.21.15.19:0/2598530899 shutdown_connections 2024-04-25T12:05:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.128+0000 7f2e38352700 1 --2- 172.21.15.19:0/2598530899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e30103480 0x7f2e3010f8d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.128+0000 7f2e38352700 1 --2- 172.21.15.19:0/2598530899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e30108a70 0x7f2e30108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.129+0000 7f2e38352700 1 --2- 172.21.15.19:0/2598530899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e30102ac0 0x7f2e30102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.129+0000 7f2e38352700 1 -- 172.21.15.19:0/2598530899 >> 172.21.15.19:0/2598530899 conn(0x7f2e30079bb0 msgr2=0x7f2e30079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.129+0000 7f2e38352700 1 -- 172.21.15.19:0/2598530899 shutdown_connections 2024-04-25T12:05:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.129+0000 7f2e38352700 1 -- 172.21.15.19:0/2598530899 wait complete. 2024-04-25T12:05:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.129+0000 7f2e38352700 1 Processor -- start 2024-04-25T12:05:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.129+0000 7f2e38352700 1 -- start start 2024-04-25T12:05:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.129+0000 7f2e38352700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e30102ac0 0x7f2e301992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.129+0000 7f2e38352700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e30103480 0x7f2e30199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:21.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.129+0000 7f2e38352700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e30108a70 0x7f2e3019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.129+0000 7f2e38352700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e30112e40 con 0x7f2e30103480 2024-04-25T12:05:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.129+0000 7f2e38352700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2e30112cc0 con 0x7f2e30108a70 2024-04-25T12:05:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.129+0000 7f2e38352700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2e30112fc0 con 0x7f2e30102ac0 2024-04-25T12:05:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e358ed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e30103480 0x7f2e30199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e358ed700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e30103480 0x7f2e30199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42784/0 (socket says 172.21.15.19:42784) 2024-04-25T12:05:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e358ed700 1 -- 172.21.15.19:0/1800803584 learned_addr learned my addr 172.21.15.19:0/1800803584 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e360ee700 1 --2- 172.21.15.19:0/1800803584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e30102ac0 0x7f2e301992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e368ef700 1 --2- 172.21.15.19:0/1800803584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e30108a70 0x7f2e3019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e358ed700 1 -- 172.21.15.19:0/1800803584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e30102ac0 msgr2=0x7f2e301992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e358ed700 1 --2- 172.21.15.19:0/1800803584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e30102ac0 0x7f2e301992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:21.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e358ed700 1 -- 172.21.15.19:0/1800803584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e30108a70 msgr2=0x7f2e3019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e358ed700 1 --2- 172.21.15.19:0/1800803584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e30108a70 0x7f2e3019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e358ed700 1 -- 172.21.15.19:0/1800803584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3019e300 con 0x7f2e30103480 2024-04-25T12:05:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e358ed700 1 --2- 172.21.15.19:0/1800803584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e30103480 0x7f2e30199830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000b530 tx=0x7f2e2000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e277fe700 1 -- 172.21.15.19:0/1800803584 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20015020 con 0x7f2e30103480 2024-04-25T12:05:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e38352700 1 -- 172.21.15.19:0/1800803584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3019e5f0 con 0x7f2e30103480 2024-04-25T12:05:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e277fe700 1 -- 172.21.15.19:0/1800803584 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2e20011d00 con 0x7f2e30103480 2024-04-25T12:05:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e277fe700 1 -- 172.21.15.19:0/1800803584 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20010e40 con 0x7f2e30103480 2024-04-25T12:05:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.130+0000 7f2e38352700 1 -- 172.21.15.19:0/1800803584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e301a5e70 con 0x7f2e30103480 2024-04-25T12:05:21.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.131+0000 7f2e38352700 1 -- 172.21.15.19:0/1800803584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e30069730 con 0x7f2e30103480 2024-04-25T12:05:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.133+0000 7f2e277fe700 1 -- 172.21.15.19:0/1800803584 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2e20018020 con 0x7f2e30103480 2024-04-25T12:05:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.133+0000 7f2e277fe700 1 --2- 172.21.15.19:0/1800803584 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2e1c0a3cd0 0x7f2e1c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.133+0000 7f2e277fe700 1 -- 172.21.15.19:0/1800803584 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f2e200cfed0 con 0x7f2e30103480 2024-04-25T12:05:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.133+0000 7f2e360ee700 1 --2- 172.21.15.19:0/1800803584 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2e1c0a3cd0 0x7f2e1c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:21.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.134+0000 7f2e360ee700 1 --2- 172.21.15.19:0/1800803584 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2e1c0a3cd0 0x7f2e1c0a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009a10 tx=0x7f2e2c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:21.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.137+0000 7f2e277fe700 1 -- 172.21.15.19:0/1800803584 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e2001f020 con 0x7f2e30103480 2024-04-25T12:05:21.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:20 smithi169 ceph-mon[32710]: pgmap v297: 97 pgs: 97 active+clean; 626 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.4 KiB/s wr, 25 op/s 2024-04-25T12:05:21.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:20 smithi150 ceph-mon[32250]: pgmap v297: 97 pgs: 97 active+clean; 626 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.4 KiB/s wr, 25 op/s 2024-04-25T12:05:21.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.345+0000 7f2e38352700 1 -- 172.21.15.19:0/1800803584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e3002df80 con 0x7f2e30103480 2024-04-25T12:05:21.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.346+0000 7f2e277fe700 1 -- 172.21.15.19:0/1800803584 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22203 (secure 0 0 0) 0x7f2e2009ca10 con 0x7f2e30103480 2024-04-25T12:05:21.344 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:05:21.344 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:05:05.642027+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:05:01.658354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:05:02.969078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:05:21.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.349+0000 7f2e38352700 1 -- 172.21.15.19:0/1800803584 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2e1c0a3cd0 msgr2=0x7f2e1c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.349+0000 7f2e38352700 1 --2- 172.21.15.19:0/1800803584 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2e1c0a3cd0 0x7f2e1c0a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009a10 tx=0x7f2e2c009190 comp rx=0 tx=0).stop 2024-04-25T12:05:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.349+0000 7f2e38352700 1 -- 172.21.15.19:0/1800803584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e30103480 msgr2=0x7f2e30199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.349+0000 7f2e38352700 1 --2- 172.21.15.19:0/1800803584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e30103480 0x7f2e30199830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000b530 tx=0x7f2e2000b840 comp rx=0 tx=0).stop 2024-04-25T12:05:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.349+0000 7f2e38352700 1 -- 172.21.15.19:0/1800803584 shutdown_connections 2024-04-25T12:05:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.349+0000 7f2e38352700 1 --2- 172.21.15.19:0/1800803584 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f2e1c0a3cd0 0x7f2e1c0a6190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.349+0000 7f2e38352700 1 --2- 172.21.15.19:0/1800803584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e30108a70 0x7f2e3019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.349+0000 7f2e38352700 1 --2- 172.21.15.19:0/1800803584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2e30102ac0 0x7f2e301992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.349+0000 7f2e38352700 1 --2- 172.21.15.19:0/1800803584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e30103480 0x7f2e30199830 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.349+0000 7f2e38352700 1 -- 172.21.15.19:0/1800803584 >> 172.21.15.19:0/1800803584 conn(0x7f2e30079bb0 msgr2=0x7f2e30101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.349+0000 7f2e38352700 1 -- 172.21.15.19:0/1800803584 shutdown_connections 2024-04-25T12:05:21.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:21.349+0000 7f2e38352700 1 -- 172.21.15.19:0/1800803584 wait complete. 2024-04-25T12:05:22.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:21 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1800803584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:22.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:21 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1800803584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:22.160 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T12:05:22.160 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T12:05:22.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:21 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1800803584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:22.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:22 smithi019 ceph-mon[29213]: pgmap v298: 97 pgs: 97 active+clean; 626 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-04-25T12:05:23.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:22 smithi169 ceph-mon[32710]: pgmap v298: 97 pgs: 97 active+clean; 626 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-04-25T12:05:23.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:22 smithi150 ceph-mon[32250]: pgmap v298: 97 pgs: 97 active+clean; 626 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-04-25T12:05:24.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.272+0000 7f1346734700 1 -- 172.21.15.19:0/3404833508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f134006afe0 msgr2=0x7f134006b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:24.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.272+0000 7f1346734700 1 --2- 172.21.15.19:0/3404833508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f134006afe0 0x7f134006b440 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1334009990 tx=0x7f1334009ca0 comp rx=0 tx=0).stop 2024-04-25T12:05:24.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.272+0000 7f1346734700 1 -- 172.21.15.19:0/3404833508 shutdown_connections 2024-04-25T12:05:24.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.272+0000 7f1346734700 1 --2- 172.21.15.19:0/3404833508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f134006b980 0x7f134010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:24.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.272+0000 7f1346734700 1 --2- 172.21.15.19:0/3404833508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1340108a20 0x7f1340108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:24.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.272+0000 7f1346734700 1 --2- 172.21.15.19:0/3404833508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f134006afe0 0x7f134006b440 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:24.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.272+0000 7f1346734700 1 -- 172.21.15.19:0/3404833508 >> 172.21.15.19:0/3404833508 conn(0x7f13400788b0 msgr2=0x7f1340078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:24.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.272+0000 7f1346734700 1 -- 172.21.15.19:0/3404833508 shutdown_connections 2024-04-25T12:05:24.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.272+0000 7f1346734700 1 -- 172.21.15.19:0/3404833508 wait complete. 2024-04-25T12:05:24.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.273+0000 7f1346734700 1 Processor -- start 2024-04-25T12:05:24.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.273+0000 7f1346734700 1 -- start start 2024-04-25T12:05:24.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.273+0000 7f1346734700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f134006afe0 0x7f13401992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:24.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.273+0000 7f1346734700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f134006b980 0x7f1340199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:24.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.273+0000 7f1346734700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340108a20 0x7f134019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:24.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.273+0000 7f1346734700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1340112d30 con 0x7f1340108a20 2024-04-25T12:05:24.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.273+0000 7f1346734700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1340112bb0 con 0x7f134006afe0 2024-04-25T12:05:24.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.273+0000 7f1346734700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f1340112eb0 con 0x7f134006b980 2024-04-25T12:05:24.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.273+0000 7f1344cd1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340108a20 0x7f134019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:24.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.273+0000 7f1344cd1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340108a20 0x7f134019db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58168/0 (socket says 172.21.15.19:58168) 2024-04-25T12:05:24.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.273+0000 7f1344cd1700 1 -- 172.21.15.19:0/3743278945 learned_addr learned my addr 172.21.15.19:0/3743278945 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:24.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.273+0000 7f133f7fe700 1 --2- 172.21.15.19:0/3743278945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f134006b980 0x7f1340199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:24.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.274+0000 7f133ffff700 1 --2- 172.21.15.19:0/3743278945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f134006afe0 0x7f13401992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:24.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.274+0000 7f1344cd1700 1 -- 172.21.15.19:0/3743278945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f134006b980 msgr2=0x7f1340199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:24.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.274+0000 7f1344cd1700 1 --2- 172.21.15.19:0/3743278945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f134006b980 0x7f1340199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:24.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.274+0000 7f1344cd1700 1 -- 172.21.15.19:0/3743278945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f134006afe0 msgr2=0x7f13401992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:24.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.274+0000 7f1344cd1700 1 --2- 172.21.15.19:0/3743278945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f134006afe0 0x7f13401992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:24.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.274+0000 7f1344cd1700 1 -- 172.21.15.19:0/3743278945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134019e2d0 con 0x7f1340108a20 2024-04-25T12:05:24.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.274+0000 7f1344cd1700 1 --2- 172.21.15.19:0/3743278945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340108a20 0x7f134019db90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1338007ee0 tx=0x7f133800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:24.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.274+0000 7f133d7fa700 1 -- 172.21.15.19:0/3743278945 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338004bb0 con 0x7f1340108a20 2024-04-25T12:05:24.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.274+0000 7f1346734700 1 -- 172.21.15.19:0/3743278945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134019e5c0 con 0x7f1340108a20 2024-04-25T12:05:24.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.274+0000 7f133d7fa700 1 -- 172.21.15.19:0/3743278945 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1338004d50 con 0x7f1340108a20 2024-04-25T12:05:24.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.274+0000 7f1346734700 1 -- 172.21.15.19:0/3743278945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13401a5e40 con 0x7f1340108a20 2024-04-25T12:05:24.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.274+0000 7f133d7fa700 1 -- 172.21.15.19:0/3743278945 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13380059e0 con 0x7f1340108a20 2024-04-25T12:05:24.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.275+0000 7f1346734700 1 -- 172.21.15.19:0/3743278945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1340104160 con 0x7f1340108a20 2024-04-25T12:05:24.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.277+0000 7f133d7fa700 1 -- 172.21.15.19:0/3743278945 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f133801a020 con 0x7f1340108a20 2024-04-25T12:05:24.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.277+0000 7f133d7fa700 1 --2- 172.21.15.19:0/3743278945 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f13280a3cd0 0x7f13280a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:24.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.278+0000 7f133ffff700 1 --2- 172.21.15.19:0/3743278945 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f13280a3cd0 0x7f13280a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:24.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.278+0000 7f133d7fa700 1 -- 172.21.15.19:0/3743278945 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f13380d01b0 con 0x7f1340108a20 2024-04-25T12:05:24.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.278+0000 7f133ffff700 1 --2- 172.21.15.19:0/3743278945 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f13280a3cd0 0x7f13280a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1330005950 tx=0x7f133000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:24.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.281+0000 7f133d7fa700 1 -- 172.21.15.19:0/3743278945 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1338010070 con 0x7f1340108a20 2024-04-25T12:05:24.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.489+0000 7f1346734700 1 -- 172.21.15.19:0/3743278945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f134002df80 con 0x7f1340108a20 2024-04-25T12:05:24.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.491+0000 7f133d7fa700 1 -- 172.21.15.19:0/3743278945 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22203 (secure 0 0 0) 0x7f133809cae0 con 0x7f1340108a20 2024-04-25T12:05:24.489 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:05:24.489 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:05:05.642027+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:05:01.658354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:05:02.969078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:05:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.493+0000 7f1346734700 1 -- 172.21.15.19:0/3743278945 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f13280a3cd0 msgr2=0x7f13280a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.493+0000 7f1346734700 1 --2- 172.21.15.19:0/3743278945 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f13280a3cd0 0x7f13280a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1330005950 tx=0x7f133000b410 comp rx=0 tx=0).stop 2024-04-25T12:05:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.493+0000 7f1346734700 1 -- 172.21.15.19:0/3743278945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340108a20 msgr2=0x7f134019db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.493+0000 7f1346734700 1 --2- 172.21.15.19:0/3743278945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340108a20 0x7f134019db90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1338007ee0 tx=0x7f133800d3b0 comp rx=0 tx=0).stop 2024-04-25T12:05:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.493+0000 7f1346734700 1 -- 172.21.15.19:0/3743278945 shutdown_connections 2024-04-25T12:05:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.493+0000 7f1346734700 1 --2- 172.21.15.19:0/3743278945 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f13280a3cd0 0x7f13280a6190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.493+0000 7f1346734700 1 --2- 172.21.15.19:0/3743278945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1340108a20 0x7f134019db90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.493+0000 7f1346734700 1 --2- 172.21.15.19:0/3743278945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f134006afe0 0x7f13401992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:24.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.493+0000 7f1346734700 1 --2- 172.21.15.19:0/3743278945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f134006b980 0x7f1340199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:24.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.493+0000 7f1346734700 1 -- 172.21.15.19:0/3743278945 >> 172.21.15.19:0/3743278945 conn(0x7f13400788b0 msgr2=0x7f13400ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:24.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.493+0000 7f1346734700 1 -- 172.21.15.19:0/3743278945 shutdown_connections 2024-04-25T12:05:24.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:24.493+0000 7f1346734700 1 -- 172.21.15.19:0/3743278945 wait complete. 2024-04-25T12:05:25.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:24 smithi019 ceph-mon[29213]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-04-25T12:05:25.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:24 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3743278945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:25.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:24 smithi169 ceph-mon[32710]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-04-25T12:05:25.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:24 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3743278945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:25.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:24 smithi150 ceph-mon[32250]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-04-25T12:05:25.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:24 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3743278945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:05:25.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T12:05:25.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T12:05:25.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T12:05:25.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T12:05:25.313 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T12:05:25.313 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T12:05:25.313 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T12:05:25.314 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T12:05:25.314 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T12:05:25.315 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T12:05:25.315 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T12:05:25.316 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T12:05:25.984 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:26.028 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:26.084 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:26.324 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:26.339 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:26.343 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:26.346 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:27.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:26 smithi019 ceph-mon[29213]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 13 op/s 2024-04-25T12:05:27.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:26 smithi169 ceph-mon[32710]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 13 op/s 2024-04-25T12:05:27.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:26 smithi150 ceph-mon[32250]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 13 op/s 2024-04-25T12:05:29.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:28 smithi019 ceph-mon[29213]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-04-25T12:05:29.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:28 smithi169 ceph-mon[32710]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-04-25T12:05:29.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:28 smithi150 ceph-mon[32250]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-04-25T12:05:31.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:30 smithi019 ceph-mon[29213]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-25T12:05:31.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:30 smithi169 ceph-mon[32710]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-25T12:05:31.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:30 smithi150 ceph-mon[32250]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-25T12:05:32.937 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:32 smithi019 ceph-mon[29213]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T12:05:33.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:32 smithi169 ceph-mon[32710]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T12:05:33.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:32 smithi150 ceph-mon[32250]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T12:05:34.909 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:34 smithi019 ceph-mon[29213]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-25T12:05:35.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:34 smithi169 ceph-mon[32710]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-25T12:05:35.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:34 smithi150 ceph-mon[32250]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-25T12:05:36.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.791+0000 7f3d3acb1700 1 -- 172.21.15.19:0/4071516412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3406bb10 msgr2=0x7f3d3410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:36.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.791+0000 7f3d3acb1700 1 --2- 172.21.15.19:0/4071516412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3406bb10 0x7f3d3410ef50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009990 tx=0x7f3d20009ca0 comp rx=0 tx=0).stop 2024-04-25T12:05:36.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.791+0000 7f3d3acb1700 1 -- 172.21.15.19:0/4071516412 shutdown_connections 2024-04-25T12:05:36.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.791+0000 7f3d3acb1700 1 --2- 172.21.15.19:0/4071516412 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d3410f490 0x7f3d34111880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:36.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.791+0000 7f3d3acb1700 1 --2- 172.21.15.19:0/4071516412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3406b1f0 0x7f3d3406b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:36.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.791+0000 7f3d3acb1700 1 --2- 172.21.15.19:0/4071516412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3406bb10 0x7f3d3410ef50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:36.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.791+0000 7f3d3acb1700 1 -- 172.21.15.19:0/4071516412 >> 172.21.15.19:0/4071516412 conn(0x7f3d340789e0 msgr2=0x7f3d34078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:36.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.791+0000 7f3d3acb1700 1 -- 172.21.15.19:0/4071516412 shutdown_connections 2024-04-25T12:05:36.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.791+0000 7f3d3acb1700 1 -- 172.21.15.19:0/4071516412 wait complete. 2024-04-25T12:05:36.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d3acb1700 1 Processor -- start 2024-04-25T12:05:36.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d3acb1700 1 -- start start 2024-04-25T12:05:36.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d3acb1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3406b1f0 0x7f3d3419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:36.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d3acb1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d3406bb10 0x7f3d3419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:36.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d3acb1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3410f490 0x7f3d3419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:36.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d3acb1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d34114fb0 con 0x7f3d3406b1f0 2024-04-25T12:05:36.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d3acb1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3d34114e30 con 0x7f3d3410f490 2024-04-25T12:05:36.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d3acb1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3d34115130 con 0x7f3d3406bb10 2024-04-25T12:05:36.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d38a4d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3406b1f0 0x7f3d3419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:36.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d38a4d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3406b1f0 0x7f3d3419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36898/0 (socket says 172.21.15.19:36898) 2024-04-25T12:05:36.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d38a4d700 1 -- 172.21.15.19:0/2707563956 learned_addr learned my addr 172.21.15.19:0/2707563956 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:36.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d3924e700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3410f490 0x7f3d3419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:36.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d33fff700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d3406bb10 0x7f3d3419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d38a4d700 1 -- 172.21.15.19:0/2707563956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d3406bb10 msgr2=0x7f3d3419b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d38a4d700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d3406bb10 0x7f3d3419b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d38a4d700 1 -- 172.21.15.19:0/2707563956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3410f490 msgr2=0x7f3d3419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d38a4d700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3410f490 0x7f3d3419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d38a4d700 1 -- 172.21.15.19:0/2707563956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d341a0370 con 0x7f3d3406b1f0 2024-04-25T12:05:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.792+0000 7f3d3924e700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d3410f490 0x7f3d3419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:05:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.793+0000 7f3d38a4d700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3406b1f0 0x7f3d3419b360 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800c9b0 tx=0x7f3d2800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.793+0000 7f3d31ffb700 1 -- 172.21.15.19:0/2707563956 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28007b00 con 0x7f3d3406b1f0 2024-04-25T12:05:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.793+0000 7f3d31ffb700 1 -- 172.21.15.19:0/2707563956 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3d28007ca0 con 0x7f3d3406b1f0 2024-04-25T12:05:36.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.793+0000 7f3d31ffb700 1 -- 172.21.15.19:0/2707563956 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d280187c0 con 0x7f3d3406b1f0 2024-04-25T12:05:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.793+0000 7f3d3acb1700 1 -- 172.21.15.19:0/2707563956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d341a0660 con 0x7f3d3406b1f0 2024-04-25T12:05:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.793+0000 7f3d3acb1700 1 -- 172.21.15.19:0/2707563956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d341a7ee0 con 0x7f3d3406b1f0 2024-04-25T12:05:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.794+0000 7f3d3acb1700 1 -- 172.21.15.19:0/2707563956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3d34111180 con 0x7f3d3406b1f0 2024-04-25T12:05:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.795+0000 7f3d31ffb700 1 -- 172.21.15.19:0/2707563956 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3d2800f450 con 0x7f3d3406b1f0 2024-04-25T12:05:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.795+0000 7f3d31ffb700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3d240a3d50 0x7f3d240a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.796+0000 7f3d33fff700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3d240a3d50 0x7f3d240a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.796+0000 7f3d31ffb700 1 -- 172.21.15.19:0/2707563956 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f3d280d0960 con 0x7f3d3406b1f0 2024-04-25T12:05:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.796+0000 7f3d31ffb700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755] conn(0x7f3d240ad530 0x7f3d240af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.796+0000 7f3d31ffb700 1 -- 172.21.15.19:0/2707563956 --> [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d240b0000 con 0x7f3d240ad530 2024-04-25T12:05:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.796+0000 7f3d31ffb700 1 -- 172.21.15.19:0/2707563956 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f3d280d0d10 con 0x7f3d3406b1f0 2024-04-25T12:05:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.796+0000 7f3d3924e700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755] conn(0x7f3d240ad530 0x7f3d240af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:36.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.796+0000 7f3d33fff700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3d240a3d50 0x7f3d240a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3d20005cd0 tx=0x7f3d20005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.796+0000 7f3d3924e700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755] conn(0x7f3d240ad530 0x7f3d240af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.797+0000 7f3d31ffb700 1 -- 172.21.15.19:0/2707563956 <== osd.0 v2:172.21.15.19:6802/86476755 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3d240b0000 con 0x7f3d240ad530 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.813+0000 7f3d3acb1700 1 -- 172.21.15.19:0/2707563956 --> [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3d34101f90 con 0x7f3d240ad530 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.814+0000 7f3d31ffb700 1 -- 172.21.15.19:0/2707563956 <== osd.0 v2:172.21.15.19:6802/86476755 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3d34101f90 con 0x7f3d240ad530 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.814+0000 7f3d3acb1700 1 -- 172.21.15.19:0/2707563956 >> [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755] conn(0x7f3d240ad530 msgr2=0x7f3d240af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.814+0000 7f3d3acb1700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755] conn(0x7f3d240ad530 0x7f3d240af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.814+0000 7f3d3acb1700 1 -- 172.21.15.19:0/2707563956 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3d240a3d50 msgr2=0x7f3d240a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.814+0000 7f3d3acb1700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3d240a3d50 0x7f3d240a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3d20005cd0 tx=0x7f3d20005c40 comp rx=0 tx=0).stop 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.814+0000 7f3d3acb1700 1 -- 172.21.15.19:0/2707563956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3406b1f0 msgr2=0x7f3d3419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.814+0000 7f3d3acb1700 1 --2- 172.21.15.19:0/2707563956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d3406b1f0 0x7f3d3419b360 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800c9b0 tx=0x7f3d2800cd70 comp rx=0 tx=0).stop 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.814+0000 7f3d3924e700 1 -- 172.21.15.19:0/2707563956 reap_dead start 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.814+0000 7f3d3acb1700 1 -- 172.21.15.19:0/2707563956 shutdown_connections 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.814+0000 7f3d3acb1700 1 -- 172.21.15.19:0/2707563956 >> 172.21.15.19:0/2707563956 conn(0x7f3d340789e0 msgr2=0x7f3d341107a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.814+0000 7f3d3acb1700 1 -- 172.21.15.19:0/2707563956 shutdown_connections 2024-04-25T12:05:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:36.814+0000 7f3d3acb1700 1 -- 172.21.15.19:0/2707563956 wait complete. 2024-04-25T12:05:37.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:36 smithi019 ceph-mon[29213]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T12:05:37.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:36 smithi169 ceph-mon[32710]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T12:05:37.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:36 smithi150 ceph-mon[32250]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T12:05:37.871 INFO:teuthology.orchestra.run.smithi019.stdout:34359738444 2024-04-25T12:05:37.871 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T12:05:39.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:38 smithi019 ceph-mon[29213]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T12:05:39.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:38 smithi169 ceph-mon[32710]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T12:05:39.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:38 smithi150 ceph-mon[32250]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T12:05:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.765+0000 7f7d38674700 1 -- 172.21.15.19:0/2808558971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3006b1f0 msgr2=0x7f7d3006b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.765+0000 7f7d38674700 1 --2- 172.21.15.19:0/2808558971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3006b1f0 0x7f7d3006b670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009990 tx=0x7f7d20009ca0 comp rx=0 tx=0).stop 2024-04-25T12:05:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.765+0000 7f7d38674700 1 -- 172.21.15.19:0/2808558971 shutdown_connections 2024-04-25T12:05:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.765+0000 7f7d38674700 1 --2- 172.21.15.19:0/2808558971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d3006bbb0 0x7f7d3010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.765+0000 7f7d38674700 1 --2- 172.21.15.19:0/2808558971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d30108a60 0x7f7d30108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.765+0000 7f7d38674700 1 --2- 172.21.15.19:0/2808558971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3006b1f0 0x7f7d3006b670 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.765+0000 7f7d38674700 1 -- 172.21.15.19:0/2808558971 >> 172.21.15.19:0/2808558971 conn(0x7f7d300789e0 msgr2=0x7f7d30078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.765+0000 7f7d38674700 1 -- 172.21.15.19:0/2808558971 shutdown_connections 2024-04-25T12:05:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.765+0000 7f7d38674700 1 -- 172.21.15.19:0/2808558971 wait complete. 2024-04-25T12:05:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.765+0000 7f7d38674700 1 Processor -- start 2024-04-25T12:05:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.765+0000 7f7d38674700 1 -- start start 2024-04-25T12:05:39.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d38674700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3006b1f0 0x7f7d301991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:39.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d38674700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d3006bbb0 0x7f7d30199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:39.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d38674700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d30108a60 0x7f7d3019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:39.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d38674700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d30112d60 con 0x7f7d3006b1f0 2024-04-25T12:05:39.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d38674700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7d30112be0 con 0x7f7d3006bbb0 2024-04-25T12:05:39.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d38674700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f7d30112ee0 con 0x7f7d30108a60 2024-04-25T12:05:39.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d36410700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3006b1f0 0x7f7d301991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:39.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d36410700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3006b1f0 0x7f7d301991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36916/0 (socket says 172.21.15.19:36916) 2024-04-25T12:05:39.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d36410700 1 -- 172.21.15.19:0/1974375037 learned_addr learned my addr 172.21.15.19:0/1974375037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:39.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d36c11700 1 --2- 172.21.15.19:0/1974375037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d30108a60 0x7f7d3019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:39.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d35c0f700 1 --2- 172.21.15.19:0/1974375037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d3006bbb0 0x7f7d30199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:39.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d36410700 1 -- 172.21.15.19:0/1974375037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d30108a60 msgr2=0x7f7d3019dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:39.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d36410700 1 --2- 172.21.15.19:0/1974375037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d30108a60 0x7f7d3019dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:39.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d36410700 1 -- 172.21.15.19:0/1974375037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d3006bbb0 msgr2=0x7f7d30199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:39.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d36410700 1 --2- 172.21.15.19:0/1974375037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d3006bbb0 0x7f7d30199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:39.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d36410700 1 -- 172.21.15.19:0/1974375037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3019e200 con 0x7f7d3006b1f0 2024-04-25T12:05:39.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.766+0000 7f7d36410700 1 --2- 172.21.15.19:0/1974375037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3006b1f0 0x7f7d301991f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00d950 tx=0x7f7d2c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:39.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.767+0000 7f7d277fe700 1 -- 172.21.15.19:0/1974375037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c016070 con 0x7f7d3006b1f0 2024-04-25T12:05:39.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.767+0000 7f7d277fe700 1 -- 172.21.15.19:0/1974375037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7d2c009e10 con 0x7f7d3006b1f0 2024-04-25T12:05:39.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.767+0000 7f7d277fe700 1 -- 172.21.15.19:0/1974375037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00f620 con 0x7f7d3006b1f0 2024-04-25T12:05:39.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.767+0000 7f7d38674700 1 -- 172.21.15.19:0/1974375037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3019e4f0 con 0x7f7d3006b1f0 2024-04-25T12:05:39.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.767+0000 7f7d38674700 1 -- 172.21.15.19:0/1974375037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d301a5d70 con 0x7f7d3006b1f0 2024-04-25T12:05:39.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.767+0000 7f7d38674700 1 -- 172.21.15.19:0/1974375037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d300631d0 con 0x7f7d3006b1f0 2024-04-25T12:05:39.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.769+0000 7f7d277fe700 1 -- 172.21.15.19:0/1974375037 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7d2c019070 con 0x7f7d3006b1f0 2024-04-25T12:05:39.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.769+0000 7f7d277fe700 1 --2- 172.21.15.19:0/1974375037 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f7d1c0a3cd0 0x7f7d1c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:39.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.770+0000 7f7d35c0f700 1 --2- 172.21.15.19:0/1974375037 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f7d1c0a3cd0 0x7f7d1c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:39.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.770+0000 7f7d277fe700 1 -- 172.21.15.19:0/1974375037 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f7d2c0d0420 con 0x7f7d3006b1f0 2024-04-25T12:05:39.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.770+0000 7f7d277fe700 1 --2- 172.21.15.19:0/1974375037 >> [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528] conn(0x7f7d1c0ad4b0 0x7f7d1c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:39.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.770+0000 7f7d277fe700 1 -- 172.21.15.19:0/1974375037 --> [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d1c0aff80 con 0x7f7d1c0ad4b0 2024-04-25T12:05:39.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.770+0000 7f7d277fe700 1 -- 172.21.15.19:0/1974375037 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f7d2c0d0900 con 0x7f7d3006b1f0 2024-04-25T12:05:39.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.770+0000 7f7d35c0f700 1 --2- 172.21.15.19:0/1974375037 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f7d1c0a3cd0 0x7f7d1c0a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006010 tx=0x7f7d2000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:39.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.770+0000 7f7d36c11700 1 --2- 172.21.15.19:0/1974375037 >> [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528] conn(0x7f7d1c0ad4b0 0x7f7d1c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:39.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.770+0000 7f7d36c11700 1 --2- 172.21.15.19:0/1974375037 >> [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528] conn(0x7f7d1c0ad4b0 0x7f7d1c0af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:39.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.771+0000 7f7d277fe700 1 -- 172.21.15.19:0/1974375037 <== osd.2 v2:172.21.15.169:6800/3641724528 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d1c0aff80 con 0x7f7d1c0ad4b0 2024-04-25T12:05:39.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.788+0000 7f7d38674700 1 -- 172.21.15.19:0/1974375037 --> [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7d30063650 con 0x7f7d1c0ad4b0 2024-04-25T12:05:39.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.789+0000 7f7d277fe700 1 -- 172.21.15.19:0/1974375037 <== osd.2 v2:172.21.15.169:6800/3641724528 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7d30063650 con 0x7f7d1c0ad4b0 2024-04-25T12:05:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.789+0000 7f7d38674700 1 -- 172.21.15.19:0/1974375037 >> [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528] conn(0x7f7d1c0ad4b0 msgr2=0x7f7d1c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.789+0000 7f7d38674700 1 --2- 172.21.15.19:0/1974375037 >> [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528] conn(0x7f7d1c0ad4b0 0x7f7d1c0af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.789+0000 7f7d38674700 1 -- 172.21.15.19:0/1974375037 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f7d1c0a3cd0 msgr2=0x7f7d1c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.789+0000 7f7d38674700 1 --2- 172.21.15.19:0/1974375037 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f7d1c0a3cd0 0x7f7d1c0a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006010 tx=0x7f7d2000b540 comp rx=0 tx=0).stop 2024-04-25T12:05:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.789+0000 7f7d38674700 1 -- 172.21.15.19:0/1974375037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3006b1f0 msgr2=0x7f7d301991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.789+0000 7f7d38674700 1 --2- 172.21.15.19:0/1974375037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3006b1f0 0x7f7d301991f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00d950 tx=0x7f7d2c00dc60 comp rx=0 tx=0).stop 2024-04-25T12:05:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.789+0000 7f7d36c11700 1 -- 172.21.15.19:0/1974375037 reap_dead start 2024-04-25T12:05:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.789+0000 7f7d38674700 1 -- 172.21.15.19:0/1974375037 shutdown_connections 2024-04-25T12:05:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.789+0000 7f7d38674700 1 -- 172.21.15.19:0/1974375037 >> 172.21.15.19:0/1974375037 conn(0x7f7d300789e0 msgr2=0x7f7d300ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.789+0000 7f7d38674700 1 -- 172.21.15.19:0/1974375037 shutdown_connections 2024-04-25T12:05:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.789+0000 7f7d38674700 1 -- 172.21.15.19:0/1974375037 wait complete. 2024-04-25T12:05:40.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.991+0000 7f5bcd63f700 1 -- 172.21.15.19:0/1855426716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8107990 msgr2=0x7f5bc8107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:40.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.991+0000 7f5bcd63f700 1 --2- 172.21.15.19:0/1855426716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8107990 0x7f5bc8107d70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000b0a0 tx=0x7f5bc000b3b0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.991+0000 7f5bcd63f700 1 -- 172.21.15.19:0/1855426716 shutdown_connections 2024-04-25T12:05:40.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.991+0000 7f5bcd63f700 1 --2- 172.21.15.19:0/1855426716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8107990 0x7f5bc8107d70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.991+0000 7f5bcd63f700 1 --2- 172.21.15.19:0/1855426716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bc8102300 0x7f5bc810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.991+0000 7f5bcd63f700 1 --2- 172.21.15.19:0/1855426716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bc8101940 0x7f5bc8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.991+0000 7f5bcd63f700 1 -- 172.21.15.19:0/1855426716 >> 172.21.15.19:0/1855426716 conn(0x7f5bc80788f0 msgr2=0x7f5bc8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:40.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.991+0000 7f5bcd63f700 1 -- 172.21.15.19:0/1855426716 shutdown_connections 2024-04-25T12:05:40.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.991+0000 7f5bcd63f700 1 -- 172.21.15.19:0/1855426716 wait complete. 2024-04-25T12:05:40.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.991+0000 7f5bcd63f700 1 Processor -- start 2024-04-25T12:05:40.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bcd63f700 1 -- start start 2024-04-25T12:05:40.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bcd63f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bc8101940 0x7f5bc819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:40.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bcd63f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bc8102300 0x7f5bc819b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:40.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bcd63f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8107990 0x7f5bc819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:40.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bcd63f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8111c40 con 0x7f5bc8107990 2024-04-25T12:05:40.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bcd63f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8111ac0 con 0x7f5bc8101940 2024-04-25T12:05:40.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bcd63f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8111dc0 con 0x7f5bc8102300 2024-04-25T12:05:40.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bc77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8107990 0x7f5bc819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:40.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bc77fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8107990 0x7f5bc819fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36932/0 (socket says 172.21.15.19:36932) 2024-04-25T12:05:40.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bc77fe700 1 -- 172.21.15.19:0/2065619174 learned_addr learned my addr 172.21.15.19:0/2065619174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:40.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bc6ffd700 1 --2- 172.21.15.19:0/2065619174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bc8101940 0x7f5bc819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:40.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bc67fc700 1 --2- 172.21.15.19:0/2065619174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bc8102300 0x7f5bc819b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:40.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bc77fe700 1 -- 172.21.15.19:0/2065619174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bc8102300 msgr2=0x7f5bc819b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bc77fe700 1 --2- 172.21.15.19:0/2065619174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bc8102300 0x7f5bc819b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bc77fe700 1 -- 172.21.15.19:0/2065619174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bc8101940 msgr2=0x7f5bc819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bc77fe700 1 --2- 172.21.15.19:0/2065619174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bc8101940 0x7f5bc819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bc77fe700 1 -- 172.21.15.19:0/2065619174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc81a03f0 con 0x7f5bc8107990 2024-04-25T12:05:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.992+0000 7f5bc77fe700 1 --2- 172.21.15.19:0/2065619174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8107990 0x7f5bc819fcb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800d980 tx=0x7f5bb800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.993+0000 7f5bbffff700 1 -- 172.21.15.19:0/2065619174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8017070 con 0x7f5bc8107990 2024-04-25T12:05:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.993+0000 7f5bbffff700 1 -- 172.21.15.19:0/2065619174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5bb8005020 con 0x7f5bc8107990 2024-04-25T12:05:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.993+0000 7f5bcd63f700 1 -- 172.21.15.19:0/2065619174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc81a06e0 con 0x7f5bc8107990 2024-04-25T12:05:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.993+0000 7f5bbffff700 1 -- 172.21.15.19:0/2065619174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8009a90 con 0x7f5bc8107990 2024-04-25T12:05:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.993+0000 7f5bcd63f700 1 -- 172.21.15.19:0/2065619174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc81a7f60 con 0x7f5bc8107990 2024-04-25T12:05:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.993+0000 7f5bcd63f700 1 -- 172.21.15.19:0/2065619174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5bc806b100 con 0x7f5bc8107990 2024-04-25T12:05:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.995+0000 7f5bbffff700 1 -- 172.21.15.19:0/2065619174 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5bb80051d0 con 0x7f5bc8107990 2024-04-25T12:05:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.995+0000 7f5bbffff700 1 --2- 172.21.15.19:0/2065619174 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5bb40a3cd0 0x7f5bb40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:40.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.996+0000 7f5bc6ffd700 1 --2- 172.21.15.19:0/2065619174 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5bb40a3cd0 0x7f5bb40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.996+0000 7f5bbffff700 1 -- 172.21.15.19:0/2065619174 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f5bb80d0730 con 0x7f5bc8107990 2024-04-25T12:05:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.996+0000 7f5bbffff700 1 --2- 172.21.15.19:0/2065619174 >> [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529] conn(0x7f5bb40ad4b0 0x7f5bb40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.996+0000 7f5bbffff700 1 -- 172.21.15.19:0/2065619174 --> [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5bb40aff80 con 0x7f5bb40ad4b0 2024-04-25T12:05:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.996+0000 7f5bbffff700 1 -- 172.21.15.19:0/2065619174 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f5bb80d0c10 con 0x7f5bc8107990 2024-04-25T12:05:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.996+0000 7f5bc6ffd700 1 --2- 172.21.15.19:0/2065619174 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5bb40a3cd0 0x7f5bb40a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009fd0 tx=0x7f5bc000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.996+0000 7f5bc67fc700 1 --2- 172.21.15.19:0/2065619174 >> [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529] conn(0x7f5bb40ad4b0 0x7f5bb40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.996+0000 7f5bc67fc700 1 --2- 172.21.15.19:0/2065619174 >> [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529] conn(0x7f5bb40ad4b0 0x7f5bb40af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:40.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:39.998+0000 7f5bbffff700 1 -- 172.21.15.19:0/2065619174 <== osd.11 v2:172.21.15.169:6824/2387365529 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bb40aff80 con 0x7f5bb40ad4b0 2024-04-25T12:05:40.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.015+0000 7f5bcd63f700 1 -- 172.21.15.19:0/2065619174 --> [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5bc8103060 con 0x7f5bb40ad4b0 2024-04-25T12:05:40.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.015+0000 7f5bbffff700 1 -- 172.21.15.19:0/2065619174 <== osd.11 v2:172.21.15.169:6824/2387365529 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5bc8103060 con 0x7f5bb40ad4b0 2024-04-25T12:05:40.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.016+0000 7f5bcd63f700 1 -- 172.21.15.19:0/2065619174 >> [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529] conn(0x7f5bb40ad4b0 msgr2=0x7f5bb40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:40.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.016+0000 7f5bcd63f700 1 --2- 172.21.15.19:0/2065619174 >> [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529] conn(0x7f5bb40ad4b0 0x7f5bb40af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.016+0000 7f5bcd63f700 1 -- 172.21.15.19:0/2065619174 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5bb40a3cd0 msgr2=0x7f5bb40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:40.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.016+0000 7f5bcd63f700 1 --2- 172.21.15.19:0/2065619174 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f5bb40a3cd0 0x7f5bb40a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009fd0 tx=0x7f5bc000b7b0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.016+0000 7f5bcd63f700 1 -- 172.21.15.19:0/2065619174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8107990 msgr2=0x7f5bc819fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:40.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.016+0000 7f5bcd63f700 1 --2- 172.21.15.19:0/2065619174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5bc8107990 0x7f5bc819fcb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800d980 tx=0x7f5bb800dc90 comp rx=0 tx=0).stop 2024-04-25T12:05:40.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.016+0000 7f5bc77fe700 1 -- 172.21.15.19:0/2065619174 reap_dead start 2024-04-25T12:05:40.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.016+0000 7f5bcd63f700 1 -- 172.21.15.19:0/2065619174 shutdown_connections 2024-04-25T12:05:40.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.016+0000 7f5bcd63f700 1 -- 172.21.15.19:0/2065619174 >> 172.21.15.19:0/2065619174 conn(0x7f5bc80788f0 msgr2=0x7f5bc806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:40.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.016+0000 7f5bcd63f700 1 -- 172.21.15.19:0/2065619174 shutdown_connections 2024-04-25T12:05:40.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.016+0000 7f5bcd63f700 1 -- 172.21.15.19:0/2065619174 wait complete. 2024-04-25T12:05:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.352+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/1323286502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eb806b0f0 msgr2=0x7f6eb810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.352+0000 7f6ebdbf6700 1 --2- 172.21.15.19:0/1323286502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eb806b0f0 0x7f6eb810f900 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000b0a0 tx=0x7f6eb000b3b0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.352+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/1323286502 shutdown_connections 2024-04-25T12:05:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.352+0000 7f6ebdbf6700 1 --2- 172.21.15.19:0/1323286502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eb806b0f0 0x7f6eb810f900 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.352+0000 7f6ebdbf6700 1 --2- 172.21.15.19:0/1323286502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb8108a30 0x7f6eb8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.352+0000 7f6ebdbf6700 1 --2- 172.21.15.19:0/1323286502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb806a750 0x7f6eb806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.352+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/1323286502 >> 172.21.15.19:0/1323286502 conn(0x7f6eb80788f0 msgr2=0x7f6eb8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.352+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/1323286502 shutdown_connections 2024-04-25T12:05:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.352+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/1323286502 wait complete. 2024-04-25T12:05:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.353+0000 7f6ebdbf6700 1 Processor -- start 2024-04-25T12:05:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.353+0000 7f6ebdbf6700 1 -- start start 2024-04-25T12:05:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.353+0000 7f6ebdbf6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb806a750 0x7f6eb8101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.353+0000 7f6ebdbf6700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eb806b0f0 0x7f6eb8101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.353+0000 7f6ebdbf6700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb8108a30 0x7f6eb8102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.353+0000 7f6ebdbf6700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb8112d90 con 0x7f6eb806b0f0 2024-04-25T12:05:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.353+0000 7f6ebdbf6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb8112c10 con 0x7f6eb806a750 2024-04-25T12:05:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.353+0000 7f6ebdbf6700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb8112f10 con 0x7f6eb8108a30 2024-04-25T12:05:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.353+0000 7f6eb6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eb806b0f0 0x7f6eb8101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6eb6ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eb806b0f0 0x7f6eb8101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36948/0 (socket says 172.21.15.19:36948) 2024-04-25T12:05:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6eb6ffd700 1 -- 172.21.15.19:0/4175659897 learned_addr learned my addr 172.21.15.19:0/4175659897 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6eb77fe700 1 --2- 172.21.15.19:0/4175659897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb806a750 0x7f6eb8101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6eb7fff700 1 --2- 172.21.15.19:0/4175659897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb8108a30 0x7f6eb8102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6eb6ffd700 1 -- 172.21.15.19:0/4175659897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb8108a30 msgr2=0x7f6eb8102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6eb6ffd700 1 --2- 172.21.15.19:0/4175659897 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6eb8108a30 0x7f6eb8102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6eb6ffd700 1 -- 172.21.15.19:0/4175659897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb806a750 msgr2=0x7f6eb8101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:40.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6eb6ffd700 1 --2- 172.21.15.19:0/4175659897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6eb806a750 0x7f6eb8101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6eb6ffd700 1 -- 172.21.15.19:0/4175659897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb81a76d0 con 0x7f6eb806b0f0 2024-04-25T12:05:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6eb6ffd700 1 --2- 172.21.15.19:0/4175659897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eb806b0f0 0x7f6eb8101f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00b530 tx=0x7f6eac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6eb4ff9700 1 -- 172.21.15.19:0/4175659897 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac015020 con 0x7f6eb806b0f0 2024-04-25T12:05:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/4175659897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb81a79c0 con 0x7f6eb806b0f0 2024-04-25T12:05:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6eb4ff9700 1 -- 172.21.15.19:0/4175659897 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6eac011d00 con 0x7f6eb806b0f0 2024-04-25T12:05:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6eb4ff9700 1 -- 172.21.15.19:0/4175659897 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac010e40 con 0x7f6eb806b0f0 2024-04-25T12:05:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.354+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/4175659897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb81a7f00 con 0x7f6eb806b0f0 2024-04-25T12:05:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.355+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/4175659897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6eb8063110 con 0x7f6eb806b0f0 2024-04-25T12:05:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.356+0000 7f6eb4ff9700 1 -- 172.21.15.19:0/4175659897 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6eac018020 con 0x7f6eb806b0f0 2024-04-25T12:05:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.357+0000 7f6eb4ff9700 1 --2- 172.21.15.19:0/4175659897 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6ea00a3cd0 0x7f6ea00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.357+0000 7f6eb77fe700 1 --2- 172.21.15.19:0/4175659897 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6ea00a3cd0 0x7f6ea00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.357+0000 7f6eb4ff9700 1 -- 172.21.15.19:0/4175659897 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f6eac0d0000 con 0x7f6eb806b0f0 2024-04-25T12:05:40.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.357+0000 7f6eb4ff9700 1 --2- 172.21.15.19:0/4175659897 >> [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081] conn(0x7f6ea00ad4b0 0x7f6ea00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:40.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.357+0000 7f6eb4ff9700 1 -- 172.21.15.19:0/4175659897 --> [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ea00aff80 con 0x7f6ea00ad4b0 2024-04-25T12:05:40.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.357+0000 7f6eb4ff9700 1 -- 172.21.15.19:0/4175659897 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f6eac0d04e0 con 0x7f6eb806b0f0 2024-04-25T12:05:40.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.358+0000 7f6eb77fe700 1 --2- 172.21.15.19:0/4175659897 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6ea00a3cd0 0x7f6ea00a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800ac50 tx=0x7f6ea800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:40.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.358+0000 7f6eb7fff700 1 --2- 172.21.15.19:0/4175659897 >> [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081] conn(0x7f6ea00ad4b0 0x7f6ea00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:40.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.358+0000 7f6eb7fff700 1 --2- 172.21.15.19:0/4175659897 >> [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081] conn(0x7f6ea00ad4b0 0x7f6ea00af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:40.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.359+0000 7f6eb4ff9700 1 -- 172.21.15.19:0/4175659897 <== osd.5 v2:172.21.15.169:6808/502865081 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6ea00aff80 con 0x7f6ea00ad4b0 2024-04-25T12:05:40.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.376+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/4175659897 --> [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6eb8063590 con 0x7f6ea00ad4b0 2024-04-25T12:05:40.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.376+0000 7f6eb4ff9700 1 -- 172.21.15.19:0/4175659897 <== osd.5 v2:172.21.15.169:6808/502865081 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6eb8063590 con 0x7f6ea00ad4b0 2024-04-25T12:05:40.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.377+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/4175659897 >> [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081] conn(0x7f6ea00ad4b0 msgr2=0x7f6ea00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:40.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.377+0000 7f6ebdbf6700 1 --2- 172.21.15.19:0/4175659897 >> [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081] conn(0x7f6ea00ad4b0 0x7f6ea00af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:40.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.377+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/4175659897 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6ea00a3cd0 msgr2=0x7f6ea00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:40.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.377+0000 7f6ebdbf6700 1 --2- 172.21.15.19:0/4175659897 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6ea00a3cd0 0x7f6ea00a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800ac50 tx=0x7f6ea800a380 comp rx=0 tx=0).stop 2024-04-25T12:05:40.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.377+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/4175659897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eb806b0f0 msgr2=0x7f6eb8101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:40.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.377+0000 7f6ebdbf6700 1 --2- 172.21.15.19:0/4175659897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eb806b0f0 0x7f6eb8101f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00b530 tx=0x7f6eac00b840 comp rx=0 tx=0).stop 2024-04-25T12:05:40.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.377+0000 7f6eb7fff700 1 -- 172.21.15.19:0/4175659897 reap_dead start 2024-04-25T12:05:40.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.377+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/4175659897 shutdown_connections 2024-04-25T12:05:40.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.377+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/4175659897 >> 172.21.15.19:0/4175659897 conn(0x7f6eb80788f0 msgr2=0x7f6eb80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:40.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.377+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/4175659897 shutdown_connections 2024-04-25T12:05:40.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:40.377+0000 7f6ebdbf6700 1 -- 172.21.15.19:0/4175659897 wait complete. 2024-04-25T12:05:41.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:40 smithi019 ceph-mon[29213]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T12:05:41.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:40 smithi169 ceph-mon[32710]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T12:05:41.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:40 smithi150 ceph-mon[32250]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T12:05:41.935 INFO:teuthology.orchestra.run.smithi019.stdout:77309411394 2024-04-25T12:05:41.936 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T12:05:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.675+0000 7f86d24e8700 1 -- 172.21.15.19:0/1980045576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86cc107990 msgr2=0x7f86cc107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.675+0000 7f86d24e8700 1 --2- 172.21.15.19:0/1980045576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86cc107990 0x7f86cc107d70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009990 tx=0x7f86bc009ca0 comp rx=0 tx=0).stop 2024-04-25T12:05:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.675+0000 7f86d24e8700 1 -- 172.21.15.19:0/1980045576 shutdown_connections 2024-04-25T12:05:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.675+0000 7f86d24e8700 1 --2- 172.21.15.19:0/1980045576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86cc107990 0x7f86cc107d70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.675+0000 7f86d24e8700 1 --2- 172.21.15.19:0/1980045576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86cc102300 0x7f86cc10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.675+0000 7f86d24e8700 1 --2- 172.21.15.19:0/1980045576 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86cc101940 0x7f86cc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.675+0000 7f86d24e8700 1 -- 172.21.15.19:0/1980045576 >> 172.21.15.19:0/1980045576 conn(0x7f86cc0788f0 msgr2=0x7f86cc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.675+0000 7f86d24e8700 1 -- 172.21.15.19:0/1980045576 shutdown_connections 2024-04-25T12:05:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.676+0000 7f86d24e8700 1 -- 172.21.15.19:0/1980045576 wait complete. 2024-04-25T12:05:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.676+0000 7f86d24e8700 1 Processor -- start 2024-04-25T12:05:42.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.676+0000 7f86d24e8700 1 -- start start 2024-04-25T12:05:42.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.676+0000 7f86d24e8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86cc101940 0x7f86cc199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:42.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.676+0000 7f86d24e8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86cc102300 0x7f86cc1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:42.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86d24e8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86cc107990 0x7f86cc19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:42.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86d24e8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc111c10 con 0x7f86cc101940 2024-04-25T12:05:42.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86d24e8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc111a90 con 0x7f86cc102300 2024-04-25T12:05:42.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86d24e8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f86cc111d90 con 0x7f86cc107990 2024-04-25T12:05:42.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86cbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86cc101940 0x7f86cc199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:42.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86cbfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86cc101940 0x7f86cc199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48198/0 (socket says 172.21.15.19:48198) 2024-04-25T12:05:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86cbfff700 1 -- 172.21.15.19:0/3990619327 learned_addr learned my addr 172.21.15.19:0/3990619327 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86cb7fe700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86cc102300 0x7f86cc1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86d0a85700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86cc107990 0x7f86cc19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86cbfff700 1 -- 172.21.15.19:0/3990619327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86cc107990 msgr2=0x7f86cc19da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86cbfff700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86cc107990 0x7f86cc19da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86cbfff700 1 -- 172.21.15.19:0/3990619327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86cc102300 msgr2=0x7f86cc1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86cbfff700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86cc102300 0x7f86cc1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86cbfff700 1 -- 172.21.15.19:0/3990619327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86cc19e170 con 0x7f86cc101940 2024-04-25T12:05:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86cb7fe700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86cc102300 0x7f86cc1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:05:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86cbfff700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86cc101940 0x7f86cc199160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f86bc005f50 tx=0x7f86bc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86c97fa700 1 -- 172.21.15.19:0/3990619327 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc026070 con 0x7f86cc101940 2024-04-25T12:05:42.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86d24e8700 1 -- 172.21.15.19:0/3990619327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86cc19e400 con 0x7f86cc101940 2024-04-25T12:05:42.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86c97fa700 1 -- 172.21.15.19:0/3990619327 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f86bc00fb40 con 0x7f86cc101940 2024-04-25T12:05:42.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.677+0000 7f86c97fa700 1 -- 172.21.15.19:0/3990619327 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0173f0 con 0x7f86cc101940 2024-04-25T12:05:42.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.678+0000 7f86d24e8700 1 -- 172.21.15.19:0/3990619327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86cc1a5ce0 con 0x7f86cc101940 2024-04-25T12:05:42.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.678+0000 7f86d24e8700 1 -- 172.21.15.19:0/3990619327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86cc105980 con 0x7f86cc101940 2024-04-25T12:05:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.680+0000 7f86c97fa700 1 -- 172.21.15.19:0/3990619327 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f86bc029070 con 0x7f86cc101940 2024-04-25T12:05:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.680+0000 7f86c97fa700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f86b80a3cd0 0x7f86b80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.680+0000 7f86cb7fe700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f86b80a3cd0 0x7f86b80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.680+0000 7f86c97fa700 1 -- 172.21.15.19:0/3990619327 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f86bc0d12d0 con 0x7f86cc101940 2024-04-25T12:05:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.681+0000 7f86c97fa700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024] conn(0x7f86b80ad4b0 0x7f86b80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.681+0000 7f86c97fa700 1 -- 172.21.15.19:0/3990619327 --> [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86bc010ea0 con 0x7f86b80ad4b0 2024-04-25T12:05:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.681+0000 7f86c97fa700 1 -- 172.21.15.19:0/3990619327 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f86bc0d17b0 con 0x7f86cc101940 2024-04-25T12:05:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.681+0000 7f86d0a85700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024] conn(0x7f86b80ad4b0 0x7f86b80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.681+0000 7f86cb7fe700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f86b80a3cd0 0x7f86b80a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86cc100120 tx=0x7f86b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.681+0000 7f86d0a85700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024] conn(0x7f86b80ad4b0 0x7f86b80af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.682+0000 7f86c97fa700 1 -- 172.21.15.19:0/3990619327 <== osd.10 v2:172.21.15.150:6824/2984224024 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86bc010ea0 con 0x7f86b80ad4b0 2024-04-25T12:05:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.699+0000 7f86d24e8700 1 -- 172.21.15.19:0/3990619327 --> [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86cc103060 con 0x7f86b80ad4b0 2024-04-25T12:05:42.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.699+0000 7f86c97fa700 1 -- 172.21.15.19:0/3990619327 <== osd.10 v2:172.21.15.150:6824/2984224024 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f86cc103060 con 0x7f86b80ad4b0 2024-04-25T12:05:42.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.699+0000 7f86d24e8700 1 -- 172.21.15.19:0/3990619327 >> [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024] conn(0x7f86b80ad4b0 msgr2=0x7f86b80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.699+0000 7f86d24e8700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024] conn(0x7f86b80ad4b0 0x7f86b80af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.699+0000 7f86d24e8700 1 -- 172.21.15.19:0/3990619327 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f86b80a3cd0 msgr2=0x7f86b80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.699+0000 7f86d24e8700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f86b80a3cd0 0x7f86b80a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f86cc100120 tx=0x7f86b4009260 comp rx=0 tx=0).stop 2024-04-25T12:05:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.699+0000 7f86d24e8700 1 -- 172.21.15.19:0/3990619327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86cc101940 msgr2=0x7f86cc199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.699+0000 7f86d24e8700 1 --2- 172.21.15.19:0/3990619327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86cc101940 0x7f86cc199160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f86bc005f50 tx=0x7f86bc0057c0 comp rx=0 tx=0).stop 2024-04-25T12:05:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.699+0000 7f86d0a85700 1 -- 172.21.15.19:0/3990619327 reap_dead start 2024-04-25T12:05:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.699+0000 7f86d24e8700 1 -- 172.21.15.19:0/3990619327 shutdown_connections 2024-04-25T12:05:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.699+0000 7f86d24e8700 1 -- 172.21.15.19:0/3990619327 >> 172.21.15.19:0/3990619327 conn(0x7f86cc0788f0 msgr2=0x7f86cc100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.700+0000 7f86d24e8700 1 -- 172.21.15.19:0/3990619327 shutdown_connections 2024-04-25T12:05:42.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:42.700+0000 7f86d24e8700 1 -- 172.21.15.19:0/3990619327 wait complete. 2024-04-25T12:05:43.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:42 smithi019 ceph-mon[29213]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T12:05:43.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:42 smithi169 ceph-mon[32710]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T12:05:43.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:42 smithi150 ceph-mon[32250]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T12:05:43.441 INFO:teuthology.orchestra.run.smithi019.stdout:343597383701 2024-04-25T12:05:43.442 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T12:05:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.780+0000 7f53efe3e700 1 -- 172.21.15.19:0/8432111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e806bb10 msgr2=0x7f53e810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.780+0000 7f53efe3e700 1 --2- 172.21.15.19:0/8432111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e806bb10 0x7f53e810f900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f53e000b0a0 tx=0x7f53e000b3b0 comp rx=0 tx=0).stop 2024-04-25T12:05:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.780+0000 7f53efe3e700 1 -- 172.21.15.19:0/8432111 shutdown_connections 2024-04-25T12:05:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.780+0000 7f53efe3e700 1 --2- 172.21.15.19:0/8432111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e806bb10 0x7f53e810f900 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.780+0000 7f53efe3e700 1 --2- 172.21.15.19:0/8432111 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e8108a30 0x7f53e8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.780+0000 7f53efe3e700 1 --2- 172.21.15.19:0/8432111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53e806b170 0x7f53e806b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.780+0000 7f53efe3e700 1 -- 172.21.15.19:0/8432111 >> 172.21.15.19:0/8432111 conn(0x7f53e80788f0 msgr2=0x7f53e8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.780+0000 7f53efe3e700 1 -- 172.21.15.19:0/8432111 shutdown_connections 2024-04-25T12:05:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.780+0000 7f53efe3e700 1 -- 172.21.15.19:0/8432111 wait complete. 2024-04-25T12:05:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.781+0000 7f53efe3e700 1 Processor -- start 2024-04-25T12:05:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.781+0000 7f53efe3e700 1 -- start start 2024-04-25T12:05:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.781+0000 7f53efe3e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e806b170 0x7f53e8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.781+0000 7f53efe3e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e806bb10 0x7f53e81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.781+0000 7f53efe3e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53e8108a30 0x7f53e819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.781+0000 7f53efe3e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8112d90 con 0x7f53e806b170 2024-04-25T12:05:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.781+0000 7f53efe3e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8112c10 con 0x7f53e8108a30 2024-04-25T12:05:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.781+0000 7f53efe3e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8112f10 con 0x7f53e806bb10 2024-04-25T12:05:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.782+0000 7f53edbda700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e806b170 0x7f53e8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.782+0000 7f53edbda700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e806b170 0x7f53e8199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48222/0 (socket says 172.21.15.19:48222) 2024-04-25T12:05:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.782+0000 7f53edbda700 1 -- 172.21.15.19:0/270733482 learned_addr learned my addr 172.21.15.19:0/270733482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.782+0000 7f53ed3d9700 1 --2- 172.21.15.19:0/270733482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e806bb10 0x7f53e81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.782+0000 7f53ee3db700 1 --2- 172.21.15.19:0/270733482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53e8108a30 0x7f53e819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.782+0000 7f53edbda700 1 -- 172.21.15.19:0/270733482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e806bb10 msgr2=0x7f53e81997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.782+0000 7f53edbda700 1 --2- 172.21.15.19:0/270733482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53e806bb10 0x7f53e81997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.782+0000 7f53edbda700 1 -- 172.21.15.19:0/270733482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53e8108a30 msgr2=0x7f53e819db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.782+0000 7f53edbda700 1 --2- 172.21.15.19:0/270733482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53e8108a30 0x7f53e819db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.782+0000 7f53edbda700 1 -- 172.21.15.19:0/270733482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e819e240 con 0x7f53e806b170 2024-04-25T12:05:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.782+0000 7f53edbda700 1 --2- 172.21.15.19:0/270733482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e806b170 0x7f53e8199280 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f53e400d980 tx=0x7f53e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.784+0000 7f53deffd700 1 -- 172.21.15.19:0/270733482 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4017070 con 0x7f53e806b170 2024-04-25T12:05:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.784+0000 7f53deffd700 1 -- 172.21.15.19:0/270733482 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f53e4005020 con 0x7f53e806b170 2024-04-25T12:05:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.784+0000 7f53efe3e700 1 -- 172.21.15.19:0/270733482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e819e530 con 0x7f53e806b170 2024-04-25T12:05:44.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.784+0000 7f53deffd700 1 -- 172.21.15.19:0/270733482 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e400fb30 con 0x7f53e806b170 2024-04-25T12:05:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.784+0000 7f53efe3e700 1 -- 172.21.15.19:0/270733482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e81a5db0 con 0x7f53e806b170 2024-04-25T12:05:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.785+0000 7f53efe3e700 1 -- 172.21.15.19:0/270733482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f53e80631d0 con 0x7f53e806b170 2024-04-25T12:05:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.786+0000 7f53deffd700 1 -- 172.21.15.19:0/270733482 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f53e400fcd0 con 0x7f53e806b170 2024-04-25T12:05:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.786+0000 7f53deffd700 1 --2- 172.21.15.19:0/270733482 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f53d40a3cd0 0x7f53d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.787+0000 7f53ed3d9700 1 --2- 172.21.15.19:0/270733482 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f53d40a3cd0 0x7f53d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:44.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.787+0000 7f53deffd700 1 -- 172.21.15.19:0/270733482 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f53e40d0230 con 0x7f53e806b170 2024-04-25T12:05:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.787+0000 7f53deffd700 1 --2- 172.21.15.19:0/270733482 >> [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254] conn(0x7f53d40ad4b0 0x7f53d40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.787+0000 7f53deffd700 1 -- 172.21.15.19:0/270733482 --> [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53e4022e50 con 0x7f53d40ad4b0 2024-04-25T12:05:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.787+0000 7f53deffd700 1 -- 172.21.15.19:0/270733482 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f53e40d7050 con 0x7f53e806b170 2024-04-25T12:05:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.787+0000 7f53ed3d9700 1 --2- 172.21.15.19:0/270733482 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f53d40a3cd0 0x7f53d40a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f53e8078f60 tx=0x7f53d8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.787+0000 7f53ee3db700 1 --2- 172.21.15.19:0/270733482 >> [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254] conn(0x7f53d40ad4b0 0x7f53d40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:44.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.787+0000 7f53ee3db700 1 --2- 172.21.15.19:0/270733482 >> [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254] conn(0x7f53d40ad4b0 0x7f53d40af8d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:44.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.788+0000 7f53deffd700 1 -- 172.21.15.19:0/270733482 <== osd.8 v2:172.21.15.169:6816/3896931254 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53e4022e50 con 0x7f53d40ad4b0 2024-04-25T12:05:44.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.809+0000 7f53efe3e700 1 -- 172.21.15.19:0/270733482 --> [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f53e8063650 con 0x7f53d40ad4b0 2024-04-25T12:05:44.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.810+0000 7f53deffd700 1 -- 172.21.15.19:0/270733482 <== osd.8 v2:172.21.15.169:6816/3896931254 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f53e8063650 con 0x7f53d40ad4b0 2024-04-25T12:05:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.810+0000 7f53efe3e700 1 -- 172.21.15.19:0/270733482 >> [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254] conn(0x7f53d40ad4b0 msgr2=0x7f53d40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.810+0000 7f53efe3e700 1 --2- 172.21.15.19:0/270733482 >> [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254] conn(0x7f53d40ad4b0 0x7f53d40af8d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.810+0000 7f53efe3e700 1 -- 172.21.15.19:0/270733482 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f53d40a3cd0 msgr2=0x7f53d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.810+0000 7f53efe3e700 1 --2- 172.21.15.19:0/270733482 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f53d40a3cd0 0x7f53d40a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f53e8078f60 tx=0x7f53d8009190 comp rx=0 tx=0).stop 2024-04-25T12:05:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.810+0000 7f53efe3e700 1 -- 172.21.15.19:0/270733482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e806b170 msgr2=0x7f53e8199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.810+0000 7f53efe3e700 1 --2- 172.21.15.19:0/270733482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53e806b170 0x7f53e8199280 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f53e400d980 tx=0x7f53e400dc90 comp rx=0 tx=0).stop 2024-04-25T12:05:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.810+0000 7f53ee3db700 1 -- 172.21.15.19:0/270733482 reap_dead start 2024-04-25T12:05:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.810+0000 7f53efe3e700 1 -- 172.21.15.19:0/270733482 shutdown_connections 2024-04-25T12:05:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.810+0000 7f53efe3e700 1 -- 172.21.15.19:0/270733482 >> 172.21.15.19:0/270733482 conn(0x7f53e80788f0 msgr2=0x7f53e80ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.810+0000 7f53efe3e700 1 -- 172.21.15.19:0/270733482 shutdown_connections 2024-04-25T12:05:44.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:44.810+0000 7f53efe3e700 1 -- 172.21.15.19:0/270733482 wait complete. 2024-04-25T12:05:45.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:44 smithi019 ceph-mon[29213]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-25T12:05:45.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:44 smithi169 ceph-mon[32710]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-25T12:05:45.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:44 smithi150 ceph-mon[32250]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-25T12:05:46.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:45 smithi019 ceph-mon[29213]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T12:05:46.155 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:45 smithi169 ceph-mon[32710]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T12:05:46.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:45 smithi150 ceph-mon[32250]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T12:05:46.837 INFO:teuthology.orchestra.run.smithi019.stdout:309237645338 2024-04-25T12:05:46.838 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T12:05:47.467 INFO:teuthology.orchestra.run.smithi019.stdout:163208757300 2024-04-25T12:05:47.467 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T12:05:48.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:47 smithi019 ceph-mon[29213]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T12:05:48.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.281+0000 7f9019f84700 1 -- 172.21.15.19:0/3959713505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9014102100 msgr2=0x7f9014102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:48.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.281+0000 7f9019f84700 1 --2- 172.21.15.19:0/3959713505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9014102100 0x7f9014102580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9008009990 tx=0x7f9008009ca0 comp rx=0 tx=0).stop 2024-04-25T12:05:48.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.281+0000 7f9019f84700 1 -- 172.21.15.19:0/3959713505 shutdown_connections 2024-04-25T12:05:48.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.281+0000 7f9019f84700 1 --2- 172.21.15.19:0/3959713505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9014102ac0 0x7f901410e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.281+0000 7f9019f84700 1 --2- 172.21.15.19:0/3959713505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9014106950 0x7f9014106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.281+0000 7f9019f84700 1 --2- 172.21.15.19:0/3959713505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9014102100 0x7f9014102580 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.281+0000 7f9019f84700 1 -- 172.21.15.19:0/3959713505 >> 172.21.15.19:0/3959713505 conn(0x7f9014077f40 msgr2=0x7f90140ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.281+0000 7f9019f84700 1 -- 172.21.15.19:0/3959713505 shutdown_connections 2024-04-25T12:05:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.281+0000 7f9019f84700 1 -- 172.21.15.19:0/3959713505 wait complete. 2024-04-25T12:05:48.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9019f84700 1 Processor -- start 2024-04-25T12:05:48.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9019f84700 1 -- start start 2024-04-25T12:05:48.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9019f84700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9014102100 0x7f90140fd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:48.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9019f84700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9014102ac0 0x7f90140fd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:48.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9019f84700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9014106950 0x7f90140fde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:48.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9019f84700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9014111d60 con 0x7f9014102ac0 2024-04-25T12:05:48.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9019f84700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9014111be0 con 0x7f9014106950 2024-04-25T12:05:48.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9019f84700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9014111ee0 con 0x7f9014102100 2024-04-25T12:05:48.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9012ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9014102ac0 0x7f90140fd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:48.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9012ffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9014102ac0 0x7f90140fd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48248/0 (socket says 172.21.15.19:48248) 2024-04-25T12:05:48.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9012ffd700 1 -- 172.21.15.19:0/1408457565 learned_addr learned my addr 172.21.15.19:0/1408457565 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:48.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9013fff700 1 --2- 172.21.15.19:0/1408457565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9014106950 0x7f90140fde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:48.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f90137fe700 1 --2- 172.21.15.19:0/1408457565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9014102100 0x7f90140fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:48.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9012ffd700 1 -- 172.21.15.19:0/1408457565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9014102100 msgr2=0x7f90140fd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:48.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9012ffd700 1 --2- 172.21.15.19:0/1408457565 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9014102100 0x7f90140fd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:48.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9012ffd700 1 -- 172.21.15.19:0/1408457565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9014106950 msgr2=0x7f90140fde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:48.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.282+0000 7f9012ffd700 1 --2- 172.21.15.19:0/1408457565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9014106950 0x7f90140fde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:48.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.283+0000 7f9012ffd700 1 -- 172.21.15.19:0/1408457565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90141a7790 con 0x7f9014102ac0 2024-04-25T12:05:48.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.283+0000 7f9012ffd700 1 --2- 172.21.15.19:0/1408457565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9014102ac0 0x7f90140fd8c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9008000c00 tx=0x7f9008009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:48.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.283+0000 7f9010ff9700 1 -- 172.21.15.19:0/1408457565 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008027070 con 0x7f9014102ac0 2024-04-25T12:05:48.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.283+0000 7f9019f84700 1 -- 172.21.15.19:0/1408457565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90141a7a20 con 0x7f9014102ac0 2024-04-25T12:05:48.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.283+0000 7f9010ff9700 1 -- 172.21.15.19:0/1408457565 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9008003fb0 con 0x7f9014102ac0 2024-04-25T12:05:48.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.283+0000 7f9010ff9700 1 -- 172.21.15.19:0/1408457565 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008022e10 con 0x7f9014102ac0 2024-04-25T12:05:48.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.283+0000 7f9019f84700 1 -- 172.21.15.19:0/1408457565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90141a7e70 con 0x7f9014102ac0 2024-04-25T12:05:48.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.284+0000 7f9019f84700 1 -- 172.21.15.19:0/1408457565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f90140630d0 con 0x7f9014102ac0 2024-04-25T12:05:48.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.285+0000 7f9010ff9700 1 -- 172.21.15.19:0/1408457565 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f900802a070 con 0x7f9014102ac0 2024-04-25T12:05:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.286+0000 7f9010ff9700 1 --2- 172.21.15.19:0/1408457565 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8ffc0a3cd0 0x7f8ffc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.286+0000 7f90137fe700 1 --2- 172.21.15.19:0/1408457565 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8ffc0a3cd0 0x7f8ffc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.286+0000 7f9010ff9700 1 -- 172.21.15.19:0/1408457565 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f90080d14b0 con 0x7f9014102ac0 2024-04-25T12:05:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.286+0000 7f9010ff9700 1 --2- 172.21.15.19:0/1408457565 >> [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887] conn(0x7f8ffc0ad4b0 0x7f8ffc0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.286+0000 7f9010ff9700 1 -- 172.21.15.19:0/1408457565 --> [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ffc0aff80 con 0x7f8ffc0ad4b0 2024-04-25T12:05:48.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.286+0000 7f9010ff9700 1 -- 172.21.15.19:0/1408457565 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f90080d1860 con 0x7f9014102ac0 2024-04-25T12:05:48.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.286+0000 7f90137fe700 1 --2- 172.21.15.19:0/1408457565 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8ffc0a3cd0 0x7f8ffc0a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9004005950 tx=0x7f900400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:48.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.286+0000 7f9013fff700 1 --2- 172.21.15.19:0/1408457565 >> [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887] conn(0x7f8ffc0ad4b0 0x7f8ffc0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:48.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.287+0000 7f9013fff700 1 --2- 172.21.15.19:0/1408457565 >> [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887] conn(0x7f8ffc0ad4b0 0x7f8ffc0af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:48.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.287+0000 7f9010ff9700 1 -- 172.21.15.19:0/1408457565 <== osd.4 v2:172.21.15.150:6808/2162522887 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ffc0aff80 con 0x7f8ffc0ad4b0 2024-04-25T12:05:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.304+0000 7f9019f84700 1 -- 172.21.15.19:0/1408457565 --> [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f90140634d0 con 0x7f8ffc0ad4b0 2024-04-25T12:05:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.305+0000 7f9010ff9700 1 -- 172.21.15.19:0/1408457565 <== osd.4 v2:172.21.15.150:6808/2162522887 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f90140634d0 con 0x7f8ffc0ad4b0 2024-04-25T12:05:48.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.305+0000 7f9019f84700 1 -- 172.21.15.19:0/1408457565 >> [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887] conn(0x7f8ffc0ad4b0 msgr2=0x7f8ffc0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:48.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.305+0000 7f9019f84700 1 --2- 172.21.15.19:0/1408457565 >> [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887] conn(0x7f8ffc0ad4b0 0x7f8ffc0af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:48.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.305+0000 7f9019f84700 1 -- 172.21.15.19:0/1408457565 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8ffc0a3cd0 msgr2=0x7f8ffc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:48.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.305+0000 7f9019f84700 1 --2- 172.21.15.19:0/1408457565 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f8ffc0a3cd0 0x7f8ffc0a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9004005950 tx=0x7f900400b410 comp rx=0 tx=0).stop 2024-04-25T12:05:48.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.305+0000 7f9019f84700 1 -- 172.21.15.19:0/1408457565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9014102ac0 msgr2=0x7f90140fd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:48.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.305+0000 7f9019f84700 1 --2- 172.21.15.19:0/1408457565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9014102ac0 0x7f90140fd8c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9008000c00 tx=0x7f9008009f50 comp rx=0 tx=0).stop 2024-04-25T12:05:48.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.305+0000 7f9013fff700 1 -- 172.21.15.19:0/1408457565 reap_dead start 2024-04-25T12:05:48.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.305+0000 7f9019f84700 1 -- 172.21.15.19:0/1408457565 shutdown_connections 2024-04-25T12:05:48.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.305+0000 7f9019f84700 1 -- 172.21.15.19:0/1408457565 >> 172.21.15.19:0/1408457565 conn(0x7f9014077f40 msgr2=0x7f90140ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:48.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.305+0000 7f9019f84700 1 -- 172.21.15.19:0/1408457565 shutdown_connections 2024-04-25T12:05:48.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.305+0000 7f9019f84700 1 -- 172.21.15.19:0/1408457565 wait complete. 2024-04-25T12:05:48.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:47 smithi169 ceph-mon[32710]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T12:05:48.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:47 smithi150 ceph-mon[32250]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T12:05:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.504+0000 7f62bcbf5700 1 -- 172.21.15.19:0/3838552563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b806b980 msgr2=0x7f62b810f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.504+0000 7f62bcbf5700 1 --2- 172.21.15.19:0/3838552563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b806b980 0x7f62b810f8f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f62b000b0a0 tx=0x7f62b000b3b0 comp rx=0 tx=0).stop 2024-04-25T12:05:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.504+0000 7f62bcbf5700 1 -- 172.21.15.19:0/3838552563 shutdown_connections 2024-04-25T12:05:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.504+0000 7f62bcbf5700 1 --2- 172.21.15.19:0/3838552563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b806b980 0x7f62b810f8f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.504+0000 7f62bcbf5700 1 --2- 172.21.15.19:0/3838552563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62b8108a20 0x7f62b8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.504+0000 7f62bcbf5700 1 --2- 172.21.15.19:0/3838552563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62b806afe0 0x7f62b806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:48.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.504+0000 7f62bcbf5700 1 -- 172.21.15.19:0/3838552563 >> 172.21.15.19:0/3838552563 conn(0x7f62b80788b0 msgr2=0x7f62b8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:48.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.504+0000 7f62bcbf5700 1 -- 172.21.15.19:0/3838552563 shutdown_connections 2024-04-25T12:05:48.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.504+0000 7f62bcbf5700 1 -- 172.21.15.19:0/3838552563 wait complete. 2024-04-25T12:05:48.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.505+0000 7f62bcbf5700 1 Processor -- start 2024-04-25T12:05:48.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.505+0000 7f62bcbf5700 1 -- start start 2024-04-25T12:05:48.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.505+0000 7f62bcbf5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b806afe0 0x7f62b8101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:48.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.505+0000 7f62bcbf5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62b806b980 0x7f62b8101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:48.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.505+0000 7f62bcbf5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62b8108a20 0x7f62b8102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:48.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.505+0000 7f62bcbf5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8112d80 con 0x7f62b806afe0 2024-04-25T12:05:48.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.505+0000 7f62bcbf5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8112c00 con 0x7f62b8108a20 2024-04-25T12:05:48.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.505+0000 7f62bcbf5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8112f00 con 0x7f62b806b980 2024-04-25T12:05:48.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.505+0000 7f62b659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b806afe0 0x7f62b8101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.505+0000 7f62b659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b806afe0 0x7f62b8101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48278/0 (socket says 172.21.15.19:48278) 2024-04-25T12:05:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.505+0000 7f62b659c700 1 -- 172.21.15.19:0/130828385 learned_addr learned my addr 172.21.15.19:0/130828385 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.505+0000 7f62b5d9b700 1 --2- 172.21.15.19:0/130828385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62b806b980 0x7f62b8101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.505+0000 7f62b6d9d700 1 --2- 172.21.15.19:0/130828385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62b8108a20 0x7f62b8102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.506+0000 7f62b659c700 1 -- 172.21.15.19:0/130828385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62b806b980 msgr2=0x7f62b8101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.506+0000 7f62b659c700 1 --2- 172.21.15.19:0/130828385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62b806b980 0x7f62b8101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.506+0000 7f62b659c700 1 -- 172.21.15.19:0/130828385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62b8108a20 msgr2=0x7f62b8102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.506+0000 7f62b659c700 1 --2- 172.21.15.19:0/130828385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62b8108a20 0x7f62b8102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.506+0000 7f62b659c700 1 -- 172.21.15.19:0/130828385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b81a77c0 con 0x7f62b806afe0 2024-04-25T12:05:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.506+0000 7f62b659c700 1 --2- 172.21.15.19:0/130828385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b806afe0 0x7f62b8101a00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f62a800d980 tx=0x7f62a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.506+0000 7f62a77fe700 1 -- 172.21.15.19:0/130828385 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8017070 con 0x7f62b806afe0 2024-04-25T12:05:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.506+0000 7f62bcbf5700 1 -- 172.21.15.19:0/130828385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b81a7ab0 con 0x7f62b806afe0 2024-04-25T12:05:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.506+0000 7f62a77fe700 1 -- 172.21.15.19:0/130828385 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f62a8005020 con 0x7f62b806afe0 2024-04-25T12:05:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.506+0000 7f62a77fe700 1 -- 172.21.15.19:0/130828385 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a800fb30 con 0x7f62b806afe0 2024-04-25T12:05:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.506+0000 7f62bcbf5700 1 -- 172.21.15.19:0/130828385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b81a7fa0 con 0x7f62b806afe0 2024-04-25T12:05:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.507+0000 7f62bcbf5700 1 -- 172.21.15.19:0/130828385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f62b8063110 con 0x7f62b806afe0 2024-04-25T12:05:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.508+0000 7f62a77fe700 1 -- 172.21.15.19:0/130828385 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f62a80051d0 con 0x7f62b806afe0 2024-04-25T12:05:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.509+0000 7f62a77fe700 1 --2- 172.21.15.19:0/130828385 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f62a00a3cd0 0x7f62a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.509+0000 7f62b5d9b700 1 --2- 172.21.15.19:0/130828385 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f62a00a3cd0 0x7f62a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.509+0000 7f62a77fe700 1 -- 172.21.15.19:0/130828385 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f62a80d06f0 con 0x7f62b806afe0 2024-04-25T12:05:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.509+0000 7f62a77fe700 1 --2- 172.21.15.19:0/130828385 >> [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664] conn(0x7f62a00ad4b0 0x7f62a00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.509+0000 7f62a77fe700 1 -- 172.21.15.19:0/130828385 --> [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62a00aff80 con 0x7f62a00ad4b0 2024-04-25T12:05:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.509+0000 7f62a77fe700 1 -- 172.21.15.19:0/130828385 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f62a80d0bd0 con 0x7f62b806afe0 2024-04-25T12:05:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.509+0000 7f62b6d9d700 1 --2- 172.21.15.19:0/130828385 >> [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664] conn(0x7f62a00ad4b0 0x7f62a00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.509+0000 7f62b5d9b700 1 --2- 172.21.15.19:0/130828385 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f62a00a3cd0 0x7f62a00a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f62b8078e20 tx=0x7f62ac00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.510+0000 7f62b6d9d700 1 --2- 172.21.15.19:0/130828385 >> [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664] conn(0x7f62a00ad4b0 0x7f62a00af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.510+0000 7f62a77fe700 1 -- 172.21.15.19:0/130828385 <== osd.9 v2:172.21.15.19:6826/3504107664 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f62a00aff80 con 0x7f62a00ad4b0 2024-04-25T12:05:48.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.527+0000 7f62bcbf5700 1 -- 172.21.15.19:0/130828385 --> [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f62b8063510 con 0x7f62a00ad4b0 2024-04-25T12:05:48.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.527+0000 7f62a77fe700 1 -- 172.21.15.19:0/130828385 <== osd.9 v2:172.21.15.19:6826/3504107664 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f62b8063510 con 0x7f62a00ad4b0 2024-04-25T12:05:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.527+0000 7f62bcbf5700 1 -- 172.21.15.19:0/130828385 >> [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664] conn(0x7f62a00ad4b0 msgr2=0x7f62a00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.527+0000 7f62bcbf5700 1 --2- 172.21.15.19:0/130828385 >> [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664] conn(0x7f62a00ad4b0 0x7f62a00af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.527+0000 7f62bcbf5700 1 -- 172.21.15.19:0/130828385 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f62a00a3cd0 msgr2=0x7f62a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.527+0000 7f62bcbf5700 1 --2- 172.21.15.19:0/130828385 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f62a00a3cd0 0x7f62a00a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f62b8078e20 tx=0x7f62ac00a300 comp rx=0 tx=0).stop 2024-04-25T12:05:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.527+0000 7f62bcbf5700 1 -- 172.21.15.19:0/130828385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b806afe0 msgr2=0x7f62b8101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.527+0000 7f62bcbf5700 1 --2- 172.21.15.19:0/130828385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b806afe0 0x7f62b8101a00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f62a800d980 tx=0x7f62a800dc90 comp rx=0 tx=0).stop 2024-04-25T12:05:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.527+0000 7f62b6d9d700 1 -- 172.21.15.19:0/130828385 reap_dead start 2024-04-25T12:05:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.528+0000 7f62bcbf5700 1 -- 172.21.15.19:0/130828385 shutdown_connections 2024-04-25T12:05:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.528+0000 7f62bcbf5700 1 -- 172.21.15.19:0/130828385 >> 172.21.15.19:0/130828385 conn(0x7f62b80788b0 msgr2=0x7f62b80ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.528+0000 7f62bcbf5700 1 -- 172.21.15.19:0/130828385 shutdown_connections 2024-04-25T12:05:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:48.528+0000 7f62bcbf5700 1 -- 172.21.15.19:0/130828385 wait complete. 2024-04-25T12:05:49.161 INFO:teuthology.orchestra.run.smithi019.stdout:141733920827 2024-04-25T12:05:49.162 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T12:05:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.443+0000 7f598b05e700 1 -- 172.21.15.19:0/1904764727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5984102a90 msgr2=0x7f5984102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.443+0000 7f598b05e700 1 --2- 172.21.15.19:0/1904764727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5984102a90 0x7f5984102f10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5974009990 tx=0x7f5974009ca0 comp rx=0 tx=0).stop 2024-04-25T12:05:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.443+0000 7f598b05e700 1 -- 172.21.15.19:0/1904764727 shutdown_connections 2024-04-25T12:05:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.443+0000 7f598b05e700 1 --2- 172.21.15.19:0/1904764727 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5984103450 0x7f598410f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.443+0000 7f598b05e700 1 --2- 172.21.15.19:0/1904764727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5984108ae0 0x7f5984108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.443+0000 7f598b05e700 1 --2- 172.21.15.19:0/1904764727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5984102a90 0x7f5984102f10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.443+0000 7f598b05e700 1 -- 172.21.15.19:0/1904764727 >> 172.21.15.19:0/1904764727 conn(0x7f5984079bf0 msgr2=0x7f598407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.443+0000 7f598b05e700 1 -- 172.21.15.19:0/1904764727 shutdown_connections 2024-04-25T12:05:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.443+0000 7f598b05e700 1 -- 172.21.15.19:0/1904764727 wait complete. 2024-04-25T12:05:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.444+0000 7f598b05e700 1 Processor -- start 2024-04-25T12:05:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.444+0000 7f598b05e700 1 -- start start 2024-04-25T12:05:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.444+0000 7f598b05e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5984102a90 0x7f5984199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.444+0000 7f598b05e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5984103450 0x7f5984199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.444+0000 7f598b05e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5984108ae0 0x7f598419daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.444+0000 7f598b05e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5984112d70 con 0x7f5984102a90 2024-04-25T12:05:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.444+0000 7f598b05e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5984112bf0 con 0x7f5984103450 2024-04-25T12:05:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.444+0000 7f598b05e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5984112ef0 con 0x7f5984108ae0 2024-04-25T12:05:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.444+0000 7f5988dfa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5984102a90 0x7f5984199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.444+0000 7f5988dfa700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5984102a90 0x7f5984199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48296/0 (socket says 172.21.15.19:48296) 2024-04-25T12:05:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.444+0000 7f5988dfa700 1 -- 172.21.15.19:0/421470120 learned_addr learned my addr 172.21.15.19:0/421470120 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.444+0000 7f5983fff700 1 --2- 172.21.15.19:0/421470120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5984103450 0x7f5984199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.444+0000 7f59895fb700 1 --2- 172.21.15.19:0/421470120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5984108ae0 0x7f598419daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.445+0000 7f5988dfa700 1 -- 172.21.15.19:0/421470120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5984108ae0 msgr2=0x7f598419daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.445+0000 7f5988dfa700 1 --2- 172.21.15.19:0/421470120 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5984108ae0 0x7f598419daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:49.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.445+0000 7f5988dfa700 1 -- 172.21.15.19:0/421470120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5984103450 msgr2=0x7f5984199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:49.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.445+0000 7f5988dfa700 1 --2- 172.21.15.19:0/421470120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5984103450 0x7f5984199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:49.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.445+0000 7f5988dfa700 1 -- 172.21.15.19:0/421470120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598419e1e0 con 0x7f5984102a90 2024-04-25T12:05:49.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.445+0000 7f5988dfa700 1 --2- 172.21.15.19:0/421470120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5984102a90 0x7f5984199220 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f596c00d950 tx=0x7f596c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:49.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.445+0000 7f5981ffb700 1 -- 172.21.15.19:0/421470120 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c016070 con 0x7f5984102a90 2024-04-25T12:05:49.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.445+0000 7f5981ffb700 1 -- 172.21.15.19:0/421470120 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f596c00fd60 con 0x7f5984102a90 2024-04-25T12:05:49.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.445+0000 7f5981ffb700 1 -- 172.21.15.19:0/421470120 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c00e610 con 0x7f5984102a90 2024-04-25T12:05:49.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.445+0000 7f598b05e700 1 -- 172.21.15.19:0/421470120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f598419e4d0 con 0x7f5984102a90 2024-04-25T12:05:49.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.445+0000 7f598b05e700 1 -- 172.21.15.19:0/421470120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59841a5d50 con 0x7f5984102a90 2024-04-25T12:05:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.446+0000 7f598b05e700 1 -- 172.21.15.19:0/421470120 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59840631d0 con 0x7f5984102a90 2024-04-25T12:05:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.447+0000 7f5981ffb700 1 -- 172.21.15.19:0/421470120 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f596c019070 con 0x7f5984102a90 2024-04-25T12:05:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.448+0000 7f5981ffb700 1 --2- 172.21.15.19:0/421470120 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f59700a3dd0 0x7f59700a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.448+0000 7f5981ffb700 1 -- 172.21.15.19:0/421470120 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f596c0d0390 con 0x7f5984102a90 2024-04-25T12:05:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.448+0000 7f5983fff700 1 --2- 172.21.15.19:0/421470120 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f59700a3dd0 0x7f59700a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.448+0000 7f5981ffb700 1 --2- 172.21.15.19:0/421470120 >> [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601] conn(0x7f59700ad5b0 0x7f59700af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.448+0000 7f5981ffb700 1 -- 172.21.15.19:0/421470120 --> [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f596c013ec0 con 0x7f59700ad5b0 2024-04-25T12:05:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.448+0000 7f5981ffb700 1 -- 172.21.15.19:0/421470120 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f596c0d0780 con 0x7f5984102a90 2024-04-25T12:05:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.448+0000 7f5983fff700 1 --2- 172.21.15.19:0/421470120 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f59700a3dd0 0x7f59700a6290 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f597400b5c0 tx=0x7f5974005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.448+0000 7f59895fb700 1 --2- 172.21.15.19:0/421470120 >> [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601] conn(0x7f59700ad5b0 0x7f59700af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.449+0000 7f59895fb700 1 --2- 172.21.15.19:0/421470120 >> [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601] conn(0x7f59700ad5b0 0x7f59700af9d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.450+0000 7f5981ffb700 1 -- 172.21.15.19:0/421470120 <== osd.1 v2:172.21.15.150:6800/1593189601 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f596c013ec0 con 0x7f59700ad5b0 2024-04-25T12:05:49.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.466+0000 7f598b05e700 1 -- 172.21.15.19:0/421470120 --> [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5984063650 con 0x7f59700ad5b0 2024-04-25T12:05:49.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.467+0000 7f5981ffb700 1 -- 172.21.15.19:0/421470120 <== osd.1 v2:172.21.15.150:6800/1593189601 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5984063650 con 0x7f59700ad5b0 2024-04-25T12:05:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.467+0000 7f598b05e700 1 -- 172.21.15.19:0/421470120 >> [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601] conn(0x7f59700ad5b0 msgr2=0x7f59700af9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.467+0000 7f598b05e700 1 --2- 172.21.15.19:0/421470120 >> [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601] conn(0x7f59700ad5b0 0x7f59700af9d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.467+0000 7f598b05e700 1 -- 172.21.15.19:0/421470120 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f59700a3dd0 msgr2=0x7f59700a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.467+0000 7f598b05e700 1 --2- 172.21.15.19:0/421470120 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f59700a3dd0 0x7f59700a6290 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f597400b5c0 tx=0x7f5974005960 comp rx=0 tx=0).stop 2024-04-25T12:05:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.467+0000 7f598b05e700 1 -- 172.21.15.19:0/421470120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5984102a90 msgr2=0x7f5984199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.467+0000 7f598b05e700 1 --2- 172.21.15.19:0/421470120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5984102a90 0x7f5984199220 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f596c00d950 tx=0x7f596c00dc60 comp rx=0 tx=0).stop 2024-04-25T12:05:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.467+0000 7f59895fb700 1 -- 172.21.15.19:0/421470120 reap_dead start 2024-04-25T12:05:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.467+0000 7f598b05e700 1 -- 172.21.15.19:0/421470120 shutdown_connections 2024-04-25T12:05:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.467+0000 7f598b05e700 1 -- 172.21.15.19:0/421470120 >> 172.21.15.19:0/421470120 conn(0x7f5984079bf0 msgr2=0x7f5984100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.467+0000 7f598b05e700 1 -- 172.21.15.19:0/421470120 shutdown_connections 2024-04-25T12:05:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:49.468+0000 7f598b05e700 1 -- 172.21.15.19:0/421470120 wait complete. 2024-04-25T12:05:49.644 INFO:teuthology.orchestra.run.smithi019.stdout:253403070501 2024-04-25T12:05:49.645 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T12:05:50.287 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:49 smithi019 ceph-mon[29213]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T12:05:50.315 INFO:teuthology.orchestra.run.smithi019.stdout:55834574922 2024-04-25T12:05:50.315 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T12:05:50.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:49 smithi169 ceph-mon[32710]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T12:05:50.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:49 smithi150 ceph-mon[32250]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T12:05:50.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.516+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2168378945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2681034b0 msgr2=0x7ff26810f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:50.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.516+0000 7ff26d8d4700 1 --2- 172.21.15.19:0/2168378945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2681034b0 0x7ff26810f8e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff26000b0a0 tx=0x7ff26000b3b0 comp rx=0 tx=0).stop 2024-04-25T12:05:50.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.516+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2168378945 shutdown_connections 2024-04-25T12:05:50.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.516+0000 7ff26d8d4700 1 --2- 172.21.15.19:0/2168378945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2681034b0 0x7ff26810f8e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:50.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.516+0000 7ff26d8d4700 1 --2- 172.21.15.19:0/2168378945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff268108a80 0x7ff268108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:50.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.516+0000 7ff26d8d4700 1 --2- 172.21.15.19:0/2168378945 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff268102af0 0x7ff268102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:50.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.516+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2168378945 >> 172.21.15.19:0/2168378945 conn(0x7ff268079ad0 msgr2=0x7ff268079ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:50.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.516+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2168378945 shutdown_connections 2024-04-25T12:05:50.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.516+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2168378945 wait complete. 2024-04-25T12:05:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.517+0000 7ff26d8d4700 1 Processor -- start 2024-04-25T12:05:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.517+0000 7ff26d8d4700 1 -- start start 2024-04-25T12:05:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.517+0000 7ff26d8d4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff268102af0 0x7ff268199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.517+0000 7ff26d8d4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2681034b0 0x7ff268199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff26d8d4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff268108a80 0x7ff26819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff26d8d4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff268112d70 con 0x7ff2681034b0 2024-04-25T12:05:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff26d8d4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff268112bf0 con 0x7ff268102af0 2024-04-25T12:05:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff26d8d4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff268112ef0 con 0x7ff268108a80 2024-04-25T12:05:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff2667fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2681034b0 0x7ff268199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff2667fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2681034b0 0x7ff268199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48300/0 (socket says 172.21.15.19:48300) 2024-04-25T12:05:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff2667fc700 1 -- 172.21.15.19:0/2289201879 learned_addr learned my addr 172.21.15.19:0/2289201879 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff266ffd700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff268102af0 0x7ff268199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff2677fe700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff268108a80 0x7ff26819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:50.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff2667fc700 1 -- 172.21.15.19:0/2289201879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff268108a80 msgr2=0x7ff26819daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff2667fc700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff268108a80 0x7ff26819daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff2667fc700 1 -- 172.21.15.19:0/2289201879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff268102af0 msgr2=0x7ff268199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff2667fc700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff268102af0 0x7ff268199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff2667fc700 1 -- 172.21.15.19:0/2289201879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26819e230 con 0x7ff2681034b0 2024-04-25T12:05:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.518+0000 7ff266ffd700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff268102af0 0x7ff268199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:05:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.519+0000 7ff2667fc700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2681034b0 0x7ff268199760 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00d950 tx=0x7ff25c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.519+0000 7ff26c8d2700 1 -- 172.21.15.19:0/2289201879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c016070 con 0x7ff2681034b0 2024-04-25T12:05:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.519+0000 7ff26c8d2700 1 -- 172.21.15.19:0/2289201879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff25c00fd60 con 0x7ff2681034b0 2024-04-25T12:05:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.519+0000 7ff26c8d2700 1 -- 172.21.15.19:0/2289201879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00e610 con 0x7ff2681034b0 2024-04-25T12:05:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.519+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2289201879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26819e520 con 0x7ff2681034b0 2024-04-25T12:05:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.520+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2289201879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff268074c70 con 0x7ff2681034b0 2024-04-25T12:05:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.520+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2289201879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff268063150 con 0x7ff2681034b0 2024-04-25T12:05:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.522+0000 7ff26c8d2700 1 -- 172.21.15.19:0/2289201879 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff25c019070 con 0x7ff2681034b0 2024-04-25T12:05:50.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.522+0000 7ff26c8d2700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff2500a3d50 0x7ff2500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:50.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.523+0000 7ff26c8d2700 1 -- 172.21.15.19:0/2289201879 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff25c0d0320 con 0x7ff2681034b0 2024-04-25T12:05:50.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.523+0000 7ff26c8d2700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913] conn(0x7ff2500ad530 0x7ff2500af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:50.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.523+0000 7ff26c8d2700 1 -- 172.21.15.19:0/2289201879 --> [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff25c024ea0 con 0x7ff2500ad530 2024-04-25T12:05:50.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.523+0000 7ff26c8d2700 1 -- 172.21.15.19:0/2289201879 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7ff25c0d0800 con 0x7ff2681034b0 2024-04-25T12:05:50.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.523+0000 7ff2677fe700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913] conn(0x7ff2500ad530 0x7ff2500af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:50.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.523+0000 7ff266ffd700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff2500a3d50 0x7ff2500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:50.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.523+0000 7ff2677fe700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913] conn(0x7ff2500ad530 0x7ff2500af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:50.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.524+0000 7ff266ffd700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff2500a3d50 0x7ff2500a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff258007f50 tx=0x7ff2580073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:50.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.524+0000 7ff26c8d2700 1 -- 172.21.15.19:0/2289201879 <== osd.3 v2:172.21.15.19:6810/3285522913 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff25c024ea0 con 0x7ff2500ad530 2024-04-25T12:05:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.542+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2289201879 --> [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff2680633e0 con 0x7ff2500ad530 2024-04-25T12:05:50.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.542+0000 7ff26c8d2700 1 -- 172.21.15.19:0/2289201879 <== osd.3 v2:172.21.15.19:6810/3285522913 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff2680633e0 con 0x7ff2500ad530 2024-04-25T12:05:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.543+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2289201879 >> [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913] conn(0x7ff2500ad530 msgr2=0x7ff2500af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.543+0000 7ff26d8d4700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913] conn(0x7ff2500ad530 0x7ff2500af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.543+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2289201879 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff2500a3d50 msgr2=0x7ff2500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.543+0000 7ff26d8d4700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff2500a3d50 0x7ff2500a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff258007f50 tx=0x7ff2580073d0 comp rx=0 tx=0).stop 2024-04-25T12:05:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.543+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2289201879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2681034b0 msgr2=0x7ff268199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.543+0000 7ff26d8d4700 1 --2- 172.21.15.19:0/2289201879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2681034b0 0x7ff268199760 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00d950 tx=0x7ff25c00dc60 comp rx=0 tx=0).stop 2024-04-25T12:05:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.543+0000 7ff2677fe700 1 -- 172.21.15.19:0/2289201879 reap_dead start 2024-04-25T12:05:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.543+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2289201879 shutdown_connections 2024-04-25T12:05:50.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.543+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2289201879 >> 172.21.15.19:0/2289201879 conn(0x7ff268079ad0 msgr2=0x7ff268101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:50.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.543+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2289201879 shutdown_connections 2024-04-25T12:05:50.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:50.543+0000 7ff26d8d4700 1 -- 172.21.15.19:0/2289201879 wait complete. 2024-04-25T12:05:51.322 INFO:teuthology.orchestra.run.smithi019.stdout:287762808865 2024-04-25T12:05:51.323 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T12:05:51.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.639+0000 7f299ca3c700 1 -- 172.21.15.19:0/1055036162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2998102300 msgr2=0x7f299810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:51.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.639+0000 7f299ca3c700 1 --2- 172.21.15.19:0/1055036162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2998102300 0x7f299810e7f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f299000b0a0 tx=0x7f299000b3b0 comp rx=0 tx=0).stop 2024-04-25T12:05:51.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.639+0000 7f299ca3c700 1 -- 172.21.15.19:0/1055036162 shutdown_connections 2024-04-25T12:05:51.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.639+0000 7f299ca3c700 1 --2- 172.21.15.19:0/1055036162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2998102300 0x7f299810e7f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:51.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.639+0000 7f299ca3c700 1 --2- 172.21.15.19:0/1055036162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2998107990 0x7f2998107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:51.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.639+0000 7f299ca3c700 1 --2- 172.21.15.19:0/1055036162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2998101940 0x7f2998101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:51.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.639+0000 7f299ca3c700 1 -- 172.21.15.19:0/1055036162 >> 172.21.15.19:0/1055036162 conn(0x7f29980788f0 msgr2=0x7f2998078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:51.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.639+0000 7f299ca3c700 1 -- 172.21.15.19:0/1055036162 shutdown_connections 2024-04-25T12:05:51.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.639+0000 7f299ca3c700 1 -- 172.21.15.19:0/1055036162 wait complete. 2024-04-25T12:05:51.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299ca3c700 1 Processor -- start 2024-04-25T12:05:51.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299ca3c700 1 -- start start 2024-04-25T12:05:51.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299ca3c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2998101940 0x7f2998198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:51.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299ca3c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2998102300 0x7f2998198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:51.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299ca3c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2998107990 0x7f299819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:51.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299ca3c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2998111c70 con 0x7f2998101940 2024-04-25T12:05:51.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299ca3c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2998111af0 con 0x7f2998107990 2024-04-25T12:05:51.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299ca3c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2998111df0 con 0x7f2998102300 2024-04-25T12:05:51.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2998101940 0x7f2998198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:51.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299659c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2998101940 0x7f2998198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48312/0 (socket says 172.21.15.19:48312) 2024-04-25T12:05:51.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299659c700 1 -- 172.21.15.19:0/4165981920 learned_addr learned my addr 172.21.15.19:0/4165981920 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:51.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f2996d9d700 1 --2- 172.21.15.19:0/4165981920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2998107990 0x7f299819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:51.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f2995d9b700 1 --2- 172.21.15.19:0/4165981920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2998102300 0x7f2998198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299659c700 1 -- 172.21.15.19:0/4165981920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2998102300 msgr2=0x7f2998198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299659c700 1 --2- 172.21.15.19:0/4165981920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2998102300 0x7f2998198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299659c700 1 -- 172.21.15.19:0/4165981920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2998107990 msgr2=0x7f299819c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299659c700 1 --2- 172.21.15.19:0/4165981920 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2998107990 0x7f299819c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.640+0000 7f299659c700 1 -- 172.21.15.19:0/4165981920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299819d050 con 0x7f2998101940 2024-04-25T12:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.641+0000 7f299659c700 1 --2- 172.21.15.19:0/4165981920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2998101940 0x7f2998198040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f298800d950 tx=0x7f298800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.641+0000 7f29877fe700 1 -- 172.21.15.19:0/4165981920 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988016070 con 0x7f2998101940 2024-04-25T12:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.641+0000 7f299ca3c700 1 -- 172.21.15.19:0/4165981920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299819d340 con 0x7f2998101940 2024-04-25T12:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.641+0000 7f29877fe700 1 -- 172.21.15.19:0/4165981920 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f298800fd60 con 0x7f2998101940 2024-04-25T12:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.641+0000 7f29877fe700 1 -- 172.21.15.19:0/4165981920 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298800e610 con 0x7f2998101940 2024-04-25T12:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.641+0000 7f299ca3c700 1 -- 172.21.15.19:0/4165981920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29981a4bc0 con 0x7f2998101940 2024-04-25T12:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.642+0000 7f299ca3c700 1 -- 172.21.15.19:0/4165981920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f29980631d0 con 0x7f2998101940 2024-04-25T12:05:51.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.643+0000 7f29877fe700 1 -- 172.21.15.19:0/4165981920 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2988019070 con 0x7f2998101940 2024-04-25T12:05:51.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.644+0000 7f29877fe700 1 --2- 172.21.15.19:0/4165981920 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f29800a3cd0 0x7f29800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:51.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.644+0000 7f2995d9b700 1 --2- 172.21.15.19:0/4165981920 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f29800a3cd0 0x7f29800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:51.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.644+0000 7f29877fe700 1 -- 172.21.15.19:0/4165981920 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f29880d02c0 con 0x7f2998101940 2024-04-25T12:05:51.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.644+0000 7f29877fe700 1 --2- 172.21.15.19:0/4165981920 >> [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084] conn(0x7f29800ad4b0 0x7f29800af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:51.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.644+0000 7f29877fe700 1 -- 172.21.15.19:0/4165981920 --> [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2988013ec0 con 0x7f29800ad4b0 2024-04-25T12:05:51.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.644+0000 7f29877fe700 1 -- 172.21.15.19:0/4165981920 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f29880d07a0 con 0x7f2998101940 2024-04-25T12:05:51.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.644+0000 7f2996d9d700 1 --2- 172.21.15.19:0/4165981920 >> [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084] conn(0x7f29800ad4b0 0x7f29800af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:51.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.644+0000 7f2995d9b700 1 --2- 172.21.15.19:0/4165981920 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f29800a3cd0 0x7f29800a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2998078ef0 tx=0x7f298c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:51.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.644+0000 7f2996d9d700 1 --2- 172.21.15.19:0/4165981920 >> [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084] conn(0x7f29800ad4b0 0x7f29800af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.645+0000 7f29877fe700 1 -- 172.21.15.19:0/4165981920 <== osd.6 v2:172.21.15.19:6818/814529084 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2988013ec0 con 0x7f29800ad4b0 2024-04-25T12:05:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.662+0000 7f299ca3c700 1 -- 172.21.15.19:0/4165981920 --> [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2998063600 con 0x7f29800ad4b0 2024-04-25T12:05:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.662+0000 7f29877fe700 1 -- 172.21.15.19:0/4165981920 <== osd.6 v2:172.21.15.19:6818/814529084 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2998063600 con 0x7f29800ad4b0 2024-04-25T12:05:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.663+0000 7f299ca3c700 1 -- 172.21.15.19:0/4165981920 >> [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084] conn(0x7f29800ad4b0 msgr2=0x7f29800af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.663+0000 7f299ca3c700 1 --2- 172.21.15.19:0/4165981920 >> [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084] conn(0x7f29800ad4b0 0x7f29800af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.663+0000 7f299ca3c700 1 -- 172.21.15.19:0/4165981920 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f29800a3cd0 msgr2=0x7f29800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.663+0000 7f299ca3c700 1 --2- 172.21.15.19:0/4165981920 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f29800a3cd0 0x7f29800a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2998078ef0 tx=0x7f298c00a300 comp rx=0 tx=0).stop 2024-04-25T12:05:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.663+0000 7f299ca3c700 1 -- 172.21.15.19:0/4165981920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2998101940 msgr2=0x7f2998198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.663+0000 7f299ca3c700 1 --2- 172.21.15.19:0/4165981920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2998101940 0x7f2998198040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f298800d950 tx=0x7f298800dc60 comp rx=0 tx=0).stop 2024-04-25T12:05:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.663+0000 7f2996d9d700 1 -- 172.21.15.19:0/4165981920 reap_dead start 2024-04-25T12:05:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.663+0000 7f299ca3c700 1 -- 172.21.15.19:0/4165981920 shutdown_connections 2024-04-25T12:05:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.663+0000 7f299ca3c700 1 -- 172.21.15.19:0/4165981920 >> 172.21.15.19:0/4165981920 conn(0x7f29980788f0 msgr2=0x7f29981005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.663+0000 7f299ca3c700 1 -- 172.21.15.19:0/4165981920 shutdown_connections 2024-04-25T12:05:51.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:51.663+0000 7f299ca3c700 1 -- 172.21.15.19:0/4165981920 wait complete. 2024-04-25T12:05:52.222 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:51 smithi019 ceph-mon[29213]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T12:05:52.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:51 smithi169 ceph-mon[32710]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T12:05:52.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:51 smithi150 ceph-mon[32250]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T12:05:52.562 INFO:teuthology.orchestra.run.smithi019.stdout:107374182462 2024-04-25T12:05:52.562 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T12:05:52.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.889+0000 7ff1d369d700 1 -- 172.21.15.19:0/572521276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1cc06bae0 msgr2=0x7ff1cc10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:52.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.889+0000 7ff1d369d700 1 --2- 172.21.15.19:0/572521276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1cc06bae0 0x7ff1cc10f930 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400b0a0 tx=0x7ff1c400b3b0 comp rx=0 tx=0).stop 2024-04-25T12:05:52.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.889+0000 7ff1d369d700 1 -- 172.21.15.19:0/572521276 shutdown_connections 2024-04-25T12:05:52.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.889+0000 7ff1d369d700 1 --2- 172.21.15.19:0/572521276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1cc06bae0 0x7ff1cc10f930 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:52.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.889+0000 7ff1d369d700 1 --2- 172.21.15.19:0/572521276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1cc108a60 0x7ff1cc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:52.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.889+0000 7ff1d369d700 1 --2- 172.21.15.19:0/572521276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1cc06b140 0x7ff1cc06b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:52.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.889+0000 7ff1d369d700 1 -- 172.21.15.19:0/572521276 >> 172.21.15.19:0/572521276 conn(0x7ff1cc078980 msgr2=0x7ff1cc078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:52.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.889+0000 7ff1d369d700 1 -- 172.21.15.19:0/572521276 shutdown_connections 2024-04-25T12:05:52.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.889+0000 7ff1d369d700 1 -- 172.21.15.19:0/572521276 wait complete. 2024-04-25T12:05:52.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.890+0000 7ff1d369d700 1 Processor -- start 2024-04-25T12:05:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.890+0000 7ff1d369d700 1 -- start start 2024-04-25T12:05:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.890+0000 7ff1d369d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1cc06b140 0x7ff1cc1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.890+0000 7ff1d369d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1cc06bae0 0x7ff1cc1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.890+0000 7ff1d369d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1cc108a60 0x7ff1cc19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.890+0000 7ff1d369d700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc112e10 con 0x7ff1cc108a60 2024-04-25T12:05:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.890+0000 7ff1d369d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc112c90 con 0x7ff1cc06bae0 2024-04-25T12:05:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.890+0000 7ff1d369d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc112f90 con 0x7ff1cc06b140 2024-04-25T12:05:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.891+0000 7ff1d1c3a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1cc108a60 0x7ff1cc19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.891+0000 7ff1d1c3a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1cc108a60 0x7ff1cc19da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53718/0 (socket says 172.21.15.19:53718) 2024-04-25T12:05:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.891+0000 7ff1d1c3a700 1 -- 172.21.15.19:0/2990633751 learned_addr learned my addr 172.21.15.19:0/2990633751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.891+0000 7ff1d0c38700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1cc06bae0 0x7ff1cc1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.891+0000 7ff1d1439700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1cc06b140 0x7ff1cc1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:52.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.891+0000 7ff1d1c3a700 1 -- 172.21.15.19:0/2990633751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1cc06b140 msgr2=0x7ff1cc1991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.891+0000 7ff1d1c3a700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1cc06b140 0x7ff1cc1991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.891+0000 7ff1d1c3a700 1 -- 172.21.15.19:0/2990633751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1cc06bae0 msgr2=0x7ff1cc1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.891+0000 7ff1d1c3a700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1cc06bae0 0x7ff1cc1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.891+0000 7ff1d1c3a700 1 -- 172.21.15.19:0/2990633751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc19e160 con 0x7ff1cc108a60 2024-04-25T12:05:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.891+0000 7ff1d0c38700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1cc06bae0 0x7ff1cc1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:05:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.891+0000 7ff1d1c3a700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1cc108a60 0x7ff1cc19da20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4006070 tx=0x7ff1c4004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.892+0000 7ff1c27fc700 1 -- 172.21.15.19:0/2990633751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4021070 con 0x7ff1cc108a60 2024-04-25T12:05:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.892+0000 7ff1c27fc700 1 -- 172.21.15.19:0/2990633751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff1c4003c00 con 0x7ff1cc108a60 2024-04-25T12:05:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.892+0000 7ff1d369d700 1 -- 172.21.15.19:0/2990633751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc19e3f0 con 0x7ff1cc108a60 2024-04-25T12:05:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.892+0000 7ff1c27fc700 1 -- 172.21.15.19:0/2990633751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c401d450 con 0x7ff1cc108a60 2024-04-25T12:05:52.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.892+0000 7ff1d369d700 1 -- 172.21.15.19:0/2990633751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc1a5cd0 con 0x7ff1cc108a60 2024-04-25T12:05:52.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.893+0000 7ff1d369d700 1 -- 172.21.15.19:0/2990633751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff1cc0631d0 con 0x7ff1cc108a60 2024-04-25T12:05:52.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.894+0000 7ff1c27fc700 1 -- 172.21.15.19:0/2990633751 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff1c4004000 con 0x7ff1cc108a60 2024-04-25T12:05:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.895+0000 7ff1c27fc700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff1b80a3d50 0x7ff1b80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.895+0000 7ff1d1439700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff1b80a3d50 0x7ff1b80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.895+0000 7ff1c27fc700 1 -- 172.21.15.19:0/2990633751 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff1c40d1790 con 0x7ff1cc108a60 2024-04-25T12:05:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.895+0000 7ff1c27fc700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658] conn(0x7ff1b80ad530 0x7ff1b80af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.895+0000 7ff1c27fc700 1 -- 172.21.15.19:0/2990633751 --> [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1c4024ec0 con 0x7ff1b80ad530 2024-04-25T12:05:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.895+0000 7ff1c27fc700 1 -- 172.21.15.19:0/2990633751 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7ff1c40d1c70 con 0x7ff1cc108a60 2024-04-25T12:05:52.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.895+0000 7ff1d1439700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff1b80a3d50 0x7ff1b80a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009bd0 tx=0x7ff1c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.896+0000 7ff1d0c38700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658] conn(0x7ff1b80ad530 0x7ff1b80af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.896+0000 7ff1d0c38700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658] conn(0x7ff1b80ad530 0x7ff1b80af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.897+0000 7ff1c27fc700 1 -- 172.21.15.19:0/2990633751 <== osd.7 v2:172.21.15.150:6816/4020549658 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1c4024ec0 con 0x7ff1b80ad530 2024-04-25T12:05:52.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.913+0000 7ff1d369d700 1 -- 172.21.15.19:0/2990633751 --> [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff1cc063650 con 0x7ff1b80ad530 2024-04-25T12:05:52.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.914+0000 7ff1c27fc700 1 -- 172.21.15.19:0/2990633751 <== osd.7 v2:172.21.15.150:6816/4020549658 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff1cc063650 con 0x7ff1b80ad530 2024-04-25T12:05:52.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.914+0000 7ff1d369d700 1 -- 172.21.15.19:0/2990633751 >> [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658] conn(0x7ff1b80ad530 msgr2=0x7ff1b80af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:52.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.914+0000 7ff1d369d700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658] conn(0x7ff1b80ad530 0x7ff1b80af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:52.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.914+0000 7ff1d369d700 1 -- 172.21.15.19:0/2990633751 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff1b80a3d50 msgr2=0x7ff1b80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:52.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.914+0000 7ff1d369d700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff1b80a3d50 0x7ff1b80a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009bd0 tx=0x7ff1c8009260 comp rx=0 tx=0).stop 2024-04-25T12:05:52.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.914+0000 7ff1d369d700 1 -- 172.21.15.19:0/2990633751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1cc108a60 msgr2=0x7ff1cc19da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.914+0000 7ff1d369d700 1 --2- 172.21.15.19:0/2990633751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1cc108a60 0x7ff1cc19da20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4006070 tx=0x7ff1c4004da0 comp rx=0 tx=0).stop 2024-04-25T12:05:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.914+0000 7ff1d1c3a700 1 -- 172.21.15.19:0/2990633751 reap_dead start 2024-04-25T12:05:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.914+0000 7ff1d369d700 1 -- 172.21.15.19:0/2990633751 shutdown_connections 2024-04-25T12:05:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.914+0000 7ff1d369d700 1 -- 172.21.15.19:0/2990633751 >> 172.21.15.19:0/2990633751 conn(0x7ff1cc078980 msgr2=0x7ff1cc0ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.914+0000 7ff1d369d700 1 -- 172.21.15.19:0/2990633751 shutdown_connections 2024-04-25T12:05:52.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:52.914+0000 7ff1d369d700 1 -- 172.21.15.19:0/2990633751 wait complete. 2024-04-25T12:05:53.557 INFO:teuthology.orchestra.run.smithi019.stdout:197568495664 2024-04-25T12:05:53.557 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T12:05:53.836 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:53.906 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:54.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:53 smithi019 ceph-mon[29213]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T12:05:54.369 INFO:teuthology.orchestra.run.smithi019.stdout:219043332140 2024-04-25T12:05:54.370 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T12:05:54.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:53 smithi169 ceph-mon[32710]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T12:05:54.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:53 smithi150 ceph-mon[32250]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T12:05:55.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-25T12:05:55.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T12:05:55.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-04-25T12:05:55.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-25T12:05:55.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-25T12:05:55.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-25T12:05:55.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-04-25T12:05:55.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T12:05:55.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-25T12:05:55.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:54 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-25T12:05:55.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-25T12:05:55.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T12:05:55.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-04-25T12:05:55.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-25T12:05:55.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-25T12:05:55.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-25T12:05:55.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-04-25T12:05:55.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T12:05:55.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-25T12:05:55.407 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:54 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-25T12:05:55.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-25T12:05:55.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T12:05:55.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-04-25T12:05:55.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-25T12:05:55.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-25T12:05:55.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-25T12:05:55.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-04-25T12:05:55.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T12:05:55.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-25T12:05:55.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:54 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-25T12:05:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:55.556 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:55.556 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:55.558 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:55.558 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:55.560 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:55.565 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:56.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:55 smithi019 ceph-mon[29213]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-25T12:05:56.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-25T12:05:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T12:05:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-04-25T12:05:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-25T12:05:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-25T12:05:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-25T12:05:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-04-25T12:05:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T12:05:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-25T12:05:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:55 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-25T12:05:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:55 smithi019 ceph-mon[29213]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T12:05:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:55 smithi169 ceph-mon[32710]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-25T12:05:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-25T12:05:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T12:05:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-04-25T12:05:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-25T12:05:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-25T12:05:56.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-25T12:05:56.407 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-04-25T12:05:56.407 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T12:05:56.407 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-25T12:05:56.407 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:55 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-25T12:05:56.407 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:55 smithi169 ceph-mon[32710]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T12:05:56.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:55 smithi150 ceph-mon[32250]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-25T12:05:56.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-25T12:05:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T12:05:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-04-25T12:05:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-25T12:05:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-25T12:05:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-25T12:05:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-04-25T12:05:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T12:05:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-25T12:05:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:55 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-25T12:05:56.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:55 smithi150 ceph-mon[32250]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T12:05:56.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.777+0000 7f367619f700 1 -- 172.21.15.19:0/235499267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36701023a0 msgr2=0x7f3670102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:56.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.777+0000 7f367619f700 1 --2- 172.21.15.19:0/235499267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36701023a0 0x7f3670102780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f36600099c0 tx=0x7f3660009cd0 comp rx=0 tx=0).stop 2024-04-25T12:05:56.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.777+0000 7f367619f700 1 -- 172.21.15.19:0/235499267 shutdown_connections 2024-04-25T12:05:56.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.777+0000 7f367619f700 1 --2- 172.21.15.19:0/235499267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36701023a0 0x7f3670102780 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:56.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.777+0000 7f367619f700 1 --2- 172.21.15.19:0/235499267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3670106cc0 0x7f367010e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:56.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.777+0000 7f367619f700 1 --2- 172.21.15.19:0/235499267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3670102d50 0x7f36701065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:56.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.777+0000 7f367619f700 1 -- 172.21.15.19:0/235499267 >> 172.21.15.19:0/235499267 conn(0x7f3670077fb0 msgr2=0x7f36700ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:56.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.777+0000 7f367619f700 1 -- 172.21.15.19:0/235499267 shutdown_connections 2024-04-25T12:05:56.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.777+0000 7f367619f700 1 -- 172.21.15.19:0/235499267 wait complete. 2024-04-25T12:05:56.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.778+0000 7f367619f700 1 Processor -- start 2024-04-25T12:05:56.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.778+0000 7f367619f700 1 -- start start 2024-04-25T12:05:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.778+0000 7f367619f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36701023a0 0x7f3670199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.778+0000 7f367619f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3670102d50 0x7f3670199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.778+0000 7f367619f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3670106cc0 0x7f367019dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.778+0000 7f367619f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3670111dd0 con 0x7f36701023a0 2024-04-25T12:05:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.778+0000 7f367619f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3670111c50 con 0x7f3670102d50 2024-04-25T12:05:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.778+0000 7f367619f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3670111f50 con 0x7f3670106cc0 2024-04-25T12:05:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.778+0000 7f366f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36701023a0 0x7f3670199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.778+0000 7f366f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36701023a0 0x7f3670199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53744/0 (socket says 172.21.15.19:53744) 2024-04-25T12:05:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.778+0000 7f366f7fe700 1 -- 172.21.15.19:0/2066257086 learned_addr learned my addr 172.21.15.19:0/2066257086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:05:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.778+0000 7f366effd700 1 --2- 172.21.15.19:0/2066257086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3670102d50 0x7f3670199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.778+0000 7f366ffff700 1 --2- 172.21.15.19:0/2066257086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3670106cc0 0x7f367019dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.779+0000 7f366f7fe700 1 -- 172.21.15.19:0/2066257086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3670106cc0 msgr2=0x7f367019dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.779+0000 7f366f7fe700 1 --2- 172.21.15.19:0/2066257086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3670106cc0 0x7f367019dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.779+0000 7f366f7fe700 1 -- 172.21.15.19:0/2066257086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3670102d50 msgr2=0x7f3670199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.779+0000 7f366f7fe700 1 --2- 172.21.15.19:0/2066257086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3670102d50 0x7f3670199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.779+0000 7f366f7fe700 1 -- 172.21.15.19:0/2066257086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367019e360 con 0x7f36701023a0 2024-04-25T12:05:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.779+0000 7f366f7fe700 1 --2- 172.21.15.19:0/2066257086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36701023a0 0x7f3670199350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f36600098e0 tx=0x7f3660005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.779+0000 7f366cff9700 1 -- 172.21.15.19:0/2066257086 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660027070 con 0x7f36701023a0 2024-04-25T12:05:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.779+0000 7f366cff9700 1 -- 172.21.15.19:0/2066257086 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3660003fb0 con 0x7f36701023a0 2024-04-25T12:05:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.779+0000 7f367619f700 1 -- 172.21.15.19:0/2066257086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367019e5f0 con 0x7f36701023a0 2024-04-25T12:05:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.779+0000 7f366cff9700 1 -- 172.21.15.19:0/2066257086 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660022e10 con 0x7f36701023a0 2024-04-25T12:05:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.779+0000 7f367619f700 1 -- 172.21.15.19:0/2066257086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36701a5ed0 con 0x7f36701023a0 2024-04-25T12:05:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.780+0000 7f367619f700 1 -- 172.21.15.19:0/2066257086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3670069730 con 0x7f36701023a0 2024-04-25T12:05:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.781+0000 7f366cff9700 1 -- 172.21.15.19:0/2066257086 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f366002a070 con 0x7f36701023a0 2024-04-25T12:05:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.782+0000 7f366cff9700 1 --2- 172.21.15.19:0/2066257086 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f36580a3d50 0x7f36580a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:05:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.782+0000 7f366effd700 1 --2- 172.21.15.19:0/2066257086 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f36580a3d50 0x7f36580a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:05:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.782+0000 7f366cff9700 1 -- 172.21.15.19:0/2066257086 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f36600d15d0 con 0x7f36701023a0 2024-04-25T12:05:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.782+0000 7f366effd700 1 --2- 172.21.15.19:0/2066257086 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f36580a3d50 0x7f36580a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3670104660 tx=0x7f3664009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:05:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:56.786+0000 7f366cff9700 1 -- 172.21.15.19:0/2066257086 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3660013070 con 0x7f36701023a0 2024-04-25T12:05:56.800 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:57.008 INFO:teuthology.orchestra.run.smithi019.stdout:163208757303 2024-04-25T12:05:57.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:57.009+0000 7f367619f700 1 -- 172.21.15.19:0/2066257086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f367019a6b0 con 0x7f36701023a0 2024-04-25T12:05:57.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:57.009+0000 7f366cff9700 1 -- 172.21.15.19:0/2066257086 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f367019a6b0 con 0x7f36701023a0 2024-04-25T12:05:57.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:57.012+0000 7f367619f700 1 -- 172.21.15.19:0/2066257086 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f36580a3d50 msgr2=0x7f36580a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:57.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:57.012+0000 7f367619f700 1 --2- 172.21.15.19:0/2066257086 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f36580a3d50 0x7f36580a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3670104660 tx=0x7f3664009190 comp rx=0 tx=0).stop 2024-04-25T12:05:57.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:57.012+0000 7f367619f700 1 -- 172.21.15.19:0/2066257086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36701023a0 msgr2=0x7f3670199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:05:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:57.012+0000 7f367619f700 1 --2- 172.21.15.19:0/2066257086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36701023a0 0x7f3670199350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f36600098e0 tx=0x7f3660005590 comp rx=0 tx=0).stop 2024-04-25T12:05:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:57.012+0000 7f367619f700 1 -- 172.21.15.19:0/2066257086 shutdown_connections 2024-04-25T12:05:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:57.012+0000 7f367619f700 1 --2- 172.21.15.19:0/2066257086 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f36580a3d50 0x7f36580a6210 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:57.012+0000 7f367619f700 1 --2- 172.21.15.19:0/2066257086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36701023a0 0x7f3670199350 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:57.012+0000 7f367619f700 1 --2- 172.21.15.19:0/2066257086 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3670106cc0 0x7f367019dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:57.012+0000 7f367619f700 1 --2- 172.21.15.19:0/2066257086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3670102d50 0x7f3670199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:05:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:57.012+0000 7f367619f700 1 -- 172.21.15.19:0/2066257086 >> 172.21.15.19:0/2066257086 conn(0x7f3670077fb0 msgr2=0x7f36701012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:05:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:57.013+0000 7f367619f700 1 -- 172.21.15.19:0/2066257086 shutdown_connections 2024-04-25T12:05:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:05:57.013+0000 7f367619f700 1 -- 172.21.15.19:0/2066257086 wait complete. 2024-04-25T12:05:57.022 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:56 smithi019 ceph-mon[29213]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T12:05:57.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:56 smithi169 ceph-mon[32710]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T12:05:57.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:56 smithi150 ceph-mon[32250]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T12:05:57.719 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:05:58.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:57 smithi019 ceph-mon[29213]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-25T12:05:58.237 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:57 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2066257086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T12:05:58.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:57 smithi169 ceph-mon[32710]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-25T12:05:58.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:57 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2066257086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T12:05:58.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:57 smithi150 ceph-mon[32250]: pgmap v318: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-25T12:05:58.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:57 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2066257086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T12:06:00.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:05:59 smithi019 ceph-mon[29213]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T12:06:00.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:05:59 smithi169 ceph-mon[32710]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T12:06:00.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:05:59 smithi150 ceph-mon[32250]: pgmap v319: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s; 83 B/s, 0 objects/s recovering 2024-04-25T12:06:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.504+0000 7f29d1944700 1 -- 172.21.15.19:0/473519668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29cc06b1f0 msgr2=0x7f29cc06b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.504+0000 7f29d1944700 1 --2- 172.21.15.19:0/473519668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29cc06b1f0 0x7f29cc06b670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009990 tx=0x7f29c0009ca0 comp rx=0 tx=0).stop 2024-04-25T12:06:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.504+0000 7f29d1944700 1 -- 172.21.15.19:0/473519668 shutdown_connections 2024-04-25T12:06:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.504+0000 7f29d1944700 1 --2- 172.21.15.19:0/473519668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29cc06bbb0 0x7f29cc10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.504+0000 7f29d1944700 1 --2- 172.21.15.19:0/473519668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc108a60 0x7f29cc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.504+0000 7f29d1944700 1 --2- 172.21.15.19:0/473519668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29cc06b1f0 0x7f29cc06b670 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.504+0000 7f29d1944700 1 -- 172.21.15.19:0/473519668 >> 172.21.15.19:0/473519668 conn(0x7f29cc0789e0 msgr2=0x7f29cc078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:01.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.504+0000 7f29d1944700 1 -- 172.21.15.19:0/473519668 shutdown_connections 2024-04-25T12:06:01.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.504+0000 7f29d1944700 1 -- 172.21.15.19:0/473519668 wait complete. 2024-04-25T12:06:01.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.505+0000 7f29d1944700 1 Processor -- start 2024-04-25T12:06:01.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.505+0000 7f29d1944700 1 -- start start 2024-04-25T12:06:01.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.505+0000 7f29d1944700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29cc06b1f0 0x7f29cc199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:01.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.505+0000 7f29d1944700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29cc06bbb0 0x7f29cc199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:01.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.505+0000 7f29d1944700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc108a60 0x7f29cc19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:01.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.505+0000 7f29d1944700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc112d60 con 0x7f29cc06b1f0 2024-04-25T12:06:01.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.505+0000 7f29d1944700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc112be0 con 0x7f29cc06bbb0 2024-04-25T12:06:01.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.505+0000 7f29d1944700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f29cc112ee0 con 0x7f29cc108a60 2024-04-25T12:06:01.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.505+0000 7f29caffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29cc06b1f0 0x7f29cc199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:01.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.505+0000 7f29caffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29cc06b1f0 0x7f29cc199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53754/0 (socket says 172.21.15.19:53754) 2024-04-25T12:06:01.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.505+0000 7f29caffd700 1 -- 172.21.15.19:0/1979864444 learned_addr learned my addr 172.21.15.19:0/1979864444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:01.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.505+0000 7f29cb7fe700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc108a60 0x7f29cc19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:01.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.505+0000 7f29ca7fc700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29cc06bbb0 0x7f29cc199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:01.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.506+0000 7f29caffd700 1 -- 172.21.15.19:0/1979864444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc108a60 msgr2=0x7f29cc19dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:01.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.506+0000 7f29caffd700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc108a60 0x7f29cc19dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:01.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.506+0000 7f29caffd700 1 -- 172.21.15.19:0/1979864444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29cc06bbb0 msgr2=0x7f29cc199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:01.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.506+0000 7f29caffd700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29cc06bbb0 0x7f29cc199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:01.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.506+0000 7f29caffd700 1 -- 172.21.15.19:0/1979864444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29cc19e1f0 con 0x7f29cc06b1f0 2024-04-25T12:06:01.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.506+0000 7f29cb7fe700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc108a60 0x7f29cc19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:01.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.506+0000 7f29ca7fc700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29cc06bbb0 0x7f29cc199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:01.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.506+0000 7f29caffd700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29cc06b1f0 0x7f29cc199230 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00d950 tx=0x7f29bc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:01.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.506+0000 7f29d0942700 1 -- 172.21.15.19:0/1979864444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc016070 con 0x7f29cc06b1f0 2024-04-25T12:06:01.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.506+0000 7f29d0942700 1 -- 172.21.15.19:0/1979864444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f29bc009b40 con 0x7f29cc06b1f0 2024-04-25T12:06:01.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.506+0000 7f29d1944700 1 -- 172.21.15.19:0/1979864444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29cc19e4e0 con 0x7f29cc06b1f0 2024-04-25T12:06:01.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.506+0000 7f29d0942700 1 -- 172.21.15.19:0/1979864444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00f620 con 0x7f29cc06b1f0 2024-04-25T12:06:01.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.506+0000 7f29d1944700 1 -- 172.21.15.19:0/1979864444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29cc1018e0 con 0x7f29cc06b1f0 2024-04-25T12:06:01.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.507+0000 7f29d1944700 1 -- 172.21.15.19:0/1979864444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29cc069730 con 0x7f29cc06b1f0 2024-04-25T12:06:01.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.508+0000 7f29d0942700 1 -- 172.21.15.19:0/1979864444 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f29bc019070 con 0x7f29cc06b1f0 2024-04-25T12:06:01.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.509+0000 7f29d0942700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f29b40a3d50 0x7f29b40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:01.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.509+0000 7f29ca7fc700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f29b40a3d50 0x7f29b40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:01.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.509+0000 7f29d0942700 1 -- 172.21.15.19:0/1979864444 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f29bc0d05b0 con 0x7f29cc06b1f0 2024-04-25T12:06:01.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.509+0000 7f29ca7fc700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f29b40a3d50 0x7f29b40a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f29c0006010 tx=0x7f29c000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:01.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.513+0000 7f29d0942700 1 -- 172.21.15.19:0/1979864444 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29bc09cdb0 con 0x7f29cc06b1f0 2024-04-25T12:06:01.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.723+0000 7f29d1944700 1 -- 172.21.15.19:0/1979864444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f29cc02df80 con 0x7f29cc06b1f0 2024-04-25T12:06:01.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.724+0000 7f29d0942700 1 -- 172.21.15.19:0/1979864444 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f29bc09c6b0 con 0x7f29cc06b1f0 2024-04-25T12:06:01.723 INFO:teuthology.orchestra.run.smithi019.stdout:77309411399 2024-04-25T12:06:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.726+0000 7f29d1944700 1 -- 172.21.15.19:0/1979864444 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f29b40a3d50 msgr2=0x7f29b40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.726+0000 7f29d1944700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f29b40a3d50 0x7f29b40a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f29c0006010 tx=0x7f29c000b540 comp rx=0 tx=0).stop 2024-04-25T12:06:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.726+0000 7f29d1944700 1 -- 172.21.15.19:0/1979864444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29cc06b1f0 msgr2=0x7f29cc199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.726+0000 7f29d1944700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29cc06b1f0 0x7f29cc199230 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00d950 tx=0x7f29bc00dd10 comp rx=0 tx=0).stop 2024-04-25T12:06:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.726+0000 7f29d1944700 1 -- 172.21.15.19:0/1979864444 shutdown_connections 2024-04-25T12:06:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.726+0000 7f29d1944700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f29b40a3d50 0x7f29b40a6210 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.726+0000 7f29d1944700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29cc06b1f0 0x7f29cc199230 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.726+0000 7f29d1944700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29cc108a60 0x7f29cc19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.726+0000 7f29d1944700 1 --2- 172.21.15.19:0/1979864444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29cc06bbb0 0x7f29cc199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:01.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.726+0000 7f29d1944700 1 -- 172.21.15.19:0/1979864444 >> 172.21.15.19:0/1979864444 conn(0x7f29cc0789e0 msgr2=0x7f29cc0ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.726+0000 7f29d1944700 1 -- 172.21.15.19:0/1979864444 shutdown_connections 2024-04-25T12:06:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:01.726+0000 7f29d1944700 1 -- 172.21.15.19:0/1979864444 wait complete. 2024-04-25T12:06:01.986 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757300 got 163208757303 for osd.5 2024-04-25T12:06:01.989 DEBUG:teuthology.parallel:result is None 2024-04-25T12:06:02.297 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:01 smithi019 ceph-mon[29213]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1 op/s; 161 B/s, 1 keys/s, 2 objects/s recovering 2024-04-25T12:06:02.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:01 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1979864444' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T12:06:02.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:01 smithi169 ceph-mon[32710]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1 op/s; 161 B/s, 1 keys/s, 2 objects/s recovering 2024-04-25T12:06:02.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:01 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1979864444' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T12:06:02.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:01 smithi150 ceph-mon[32250]: pgmap v320: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1 op/s; 161 B/s, 1 keys/s, 2 objects/s recovering 2024-04-25T12:06:02.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:01 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1979864444' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T12:06:03.141 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411399 for osd.2 2024-04-25T12:06:03.141 DEBUG:teuthology.parallel:result is None 2024-04-25T12:06:03.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:03 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:06:03.405 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:03 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:06:03.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:03 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:06:04.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:04 smithi019 ceph-mon[29213]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4 op/s; 161 B/s, 1 keys/s, 2 objects/s recovering 2024-04-25T12:06:04.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:04 smithi169 ceph-mon[32710]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4 op/s; 161 B/s, 1 keys/s, 2 objects/s recovering 2024-04-25T12:06:04.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:04 smithi150 ceph-mon[32250]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4 op/s; 161 B/s, 1 keys/s, 2 objects/s recovering 2024-04-25T12:06:05.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:05 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:05.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:05 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:05.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:05 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:05.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:05 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:05.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:05 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:05.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:05 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:05.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:05 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:05.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:05 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:05.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:05 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:05.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:05 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:05.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:05 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:05.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:05 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:06.856 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:06 smithi019 ceph-mon[29213]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4 op/s; 130 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T12:06:06.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:06 smithi169 ceph-mon[32710]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4 op/s; 130 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T12:06:06.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:06 smithi150 ceph-mon[32250]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4 op/s; 130 B/s, 1 keys/s, 1 objects/s recovering 2024-04-25T12:06:07.269 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:06:07.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.968+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/1115956196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c04108a20 msgr2=0x7f9c04108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:07.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.968+0000 7f9c0c1d2700 1 --2- 172.21.15.19:0/1115956196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c04108a20 0x7f9c04108e00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009990 tx=0x7f9c00009ca0 comp rx=0 tx=0).stop 2024-04-25T12:06:07.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.968+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/1115956196 shutdown_connections 2024-04-25T12:06:07.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.968+0000 7f9c0c1d2700 1 --2- 172.21.15.19:0/1115956196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c04108a20 0x7f9c04108e00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:07.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.968+0000 7f9c0c1d2700 1 --2- 172.21.15.19:0/1115956196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c0406b980 0x7f9c0410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:07.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.968+0000 7f9c0c1d2700 1 --2- 172.21.15.19:0/1115956196 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c0406afe0 0x7f9c0406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:07.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.968+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/1115956196 >> 172.21.15.19:0/1115956196 conn(0x7f9c040788b0 msgr2=0x7f9c04078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:07.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.968+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/1115956196 shutdown_connections 2024-04-25T12:06:07.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.968+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/1115956196 wait complete. 2024-04-25T12:06:07.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.968+0000 7f9c0c1d2700 1 Processor -- start 2024-04-25T12:06:07.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.968+0000 7f9c0c1d2700 1 -- start start 2024-04-25T12:06:07.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c0c1d2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c0406afe0 0x7f9c04101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:07.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c0c1d2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c0406b980 0x7f9c04101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:07.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c0c1d2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c04108a20 0x7f9c04102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:07.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c0c1d2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04112d40 con 0x7f9c04108a20 2024-04-25T12:06:07.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c0c1d2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04112bc0 con 0x7f9c0406b980 2024-04-25T12:06:07.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c0c1d2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04112ec0 con 0x7f9c0406afe0 2024-04-25T12:06:07.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c0a76f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c04108a20 0x7f9c04102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:07.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c0a76f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c04108a20 0x7f9c04102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57760/0 (socket says 172.21.15.19:57760) 2024-04-25T12:06:07.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c0a76f700 1 -- 172.21.15.19:0/3174132082 learned_addr learned my addr 172.21.15.19:0/3174132082 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:07.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c09f6e700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c0406afe0 0x7f9c04101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:07.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c0976d700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c0406b980 0x7f9c04101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:07.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c09f6e700 1 -- 172.21.15.19:0/3174132082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c0406b980 msgr2=0x7f9c04101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:07.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c09f6e700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c0406b980 0x7f9c04101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:07.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c09f6e700 1 -- 172.21.15.19:0/3174132082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c04108a20 msgr2=0x7f9c04102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:07.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c09f6e700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c04108a20 0x7f9c04102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:07.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c09f6e700 1 -- 172.21.15.19:0/3174132082 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c041a7760 con 0x7f9c0406afe0 2024-04-25T12:06:07.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c0a76f700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c04108a20 0x7f9c04102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:06:07.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.969+0000 7f9c09f6e700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c0406afe0 0x7f9c04101a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9c000059d0 tx=0x7f9c000047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.970+0000 7f9bfaffd700 1 -- 172.21.15.19:0/3174132082 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00027070 con 0x7f9c0406afe0 2024-04-25T12:06:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.970+0000 7f9bfaffd700 1 -- 172.21.15.19:0/3174132082 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9c0000b610 con 0x7f9c0406afe0 2024-04-25T12:06:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.970+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/3174132082 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c041a79f0 con 0x7f9c0406afe0 2024-04-25T12:06:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.970+0000 7f9bfaffd700 1 -- 172.21.15.19:0/3174132082 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c000173f0 con 0x7f9c0406afe0 2024-04-25T12:06:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.970+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/3174132082 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c041a7e40 con 0x7f9c0406afe0 2024-04-25T12:06:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.971+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/3174132082 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c04069730 con 0x7f9c0406afe0 2024-04-25T12:06:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.972+0000 7f9bfaffd700 1 -- 172.21.15.19:0/3174132082 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9c0002a070 con 0x7f9c0406afe0 2024-04-25T12:06:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.972+0000 7f9bfaffd700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9bf00a3cd0 0x7f9bf00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.972+0000 7f9bfaffd700 1 -- 172.21.15.19:0/3174132082 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9c000d1bb0 con 0x7f9c0406afe0 2024-04-25T12:06:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.972+0000 7f9c0976d700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9bf00a3cd0 0x7f9bf00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:07.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.973+0000 7f9c0976d700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9bf00a3cd0 0x7f9bf00a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9c04078f90 tx=0x7f9bf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:07.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:07.976+0000 7f9bfaffd700 1 -- 172.21.15.19:0/3174132082 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c00031070 con 0x7f9c0406afe0 2024-04-25T12:06:08.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:08.186+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/3174132082 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9c0402e1e0 con 0x7f9c0406afe0 2024-04-25T12:06:08.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:08.186+0000 7f9bfaffd700 1 -- 172.21.15.19:0/3174132082 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9c00013070 con 0x7f9c0406afe0 2024-04-25T12:06:08.185 INFO:teuthology.orchestra.run.smithi019.stdout:55834574925 2024-04-25T12:06:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:08.188+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/3174132082 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9bf00a3cd0 msgr2=0x7f9bf00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:08.188+0000 7f9c0c1d2700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9bf00a3cd0 0x7f9bf00a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9c04078f90 tx=0x7f9bf4009260 comp rx=0 tx=0).stop 2024-04-25T12:06:08.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:08.188+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/3174132082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c0406afe0 msgr2=0x7f9c04101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:08.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:08.188+0000 7f9c0c1d2700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c0406afe0 0x7f9c04101a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9c000059d0 tx=0x7f9c000047b0 comp rx=0 tx=0).stop 2024-04-25T12:06:08.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:08.188+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/3174132082 shutdown_connections 2024-04-25T12:06:08.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:08.188+0000 7f9c0c1d2700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9bf00a3cd0 0x7f9bf00a6190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:08.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:08.188+0000 7f9c0c1d2700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c04108a20 0x7f9c04102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:08.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:08.188+0000 7f9c0c1d2700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c0406afe0 0x7f9c04101a00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:08.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:08.188+0000 7f9c0c1d2700 1 --2- 172.21.15.19:0/3174132082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c0406b980 0x7f9c04101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:08.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:08.188+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/3174132082 >> 172.21.15.19:0/3174132082 conn(0x7f9c040788b0 msgr2=0x7f9c04100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:08.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:08.189+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/3174132082 shutdown_connections 2024-04-25T12:06:08.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:08.189+0000 7f9c0c1d2700 1 -- 172.21.15.19:0/3174132082 wait complete. 2024-04-25T12:06:08.842 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:08 smithi019 ceph-mon[29213]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6 op/s; 118 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T12:06:08.842 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:08 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3174132082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T12:06:08.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:08 smithi169 ceph-mon[32710]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6 op/s; 118 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T12:06:08.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:08 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3174132082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T12:06:08.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:08 smithi150 ceph-mon[32250]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6 op/s; 118 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T12:06:08.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:08 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3174132082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T12:06:09.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.412+0000 7f675ba77700 1 -- 172.21.15.19:0/1177364083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6754102ac0 msgr2=0x7f6754102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:09.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.412+0000 7f675ba77700 1 --2- 172.21.15.19:0/1177364083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6754102ac0 0x7f6754102f40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6750009990 tx=0x7f6750009ca0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.412+0000 7f675ba77700 1 -- 172.21.15.19:0/1177364083 shutdown_connections 2024-04-25T12:06:09.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.412+0000 7f675ba77700 1 --2- 172.21.15.19:0/1177364083 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6754103480 0x7f675410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.412+0000 7f675ba77700 1 --2- 172.21.15.19:0/1177364083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6754108ab0 0x7f6754108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.412+0000 7f675ba77700 1 --2- 172.21.15.19:0/1177364083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6754102ac0 0x7f6754102f40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.412+0000 7f675ba77700 1 -- 172.21.15.19:0/1177364083 >> 172.21.15.19:0/1177364083 conn(0x7f6754079bf0 msgr2=0x7f675407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:09.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.412+0000 7f675ba77700 1 -- 172.21.15.19:0/1177364083 shutdown_connections 2024-04-25T12:06:09.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.412+0000 7f675ba77700 1 -- 172.21.15.19:0/1177364083 wait complete. 2024-04-25T12:06:09.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.413+0000 7f675ba77700 1 Processor -- start 2024-04-25T12:06:09.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.413+0000 7f675ba77700 1 -- start start 2024-04-25T12:06:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.413+0000 7f675ba77700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6754102ac0 0x7f6754199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.413+0000 7f675ba77700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6754103480 0x7f6754199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.413+0000 7f675ba77700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6754108ab0 0x7f675419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.413+0000 7f675ba77700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6754112dd0 con 0x7f6754108ab0 2024-04-25T12:06:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.413+0000 7f675ba77700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6754112c50 con 0x7f6754103480 2024-04-25T12:06:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.413+0000 7f675ba77700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6754112f50 con 0x7f6754102ac0 2024-04-25T12:06:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.413+0000 7f675a014700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6754108ab0 0x7f675419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f675a014700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6754108ab0 0x7f675419daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57792/0 (socket says 172.21.15.19:57792) 2024-04-25T12:06:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f675a014700 1 -- 172.21.15.19:0/3168884744 learned_addr learned my addr 172.21.15.19:0/3168884744 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f6759012700 1 --2- 172.21.15.19:0/3168884744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6754103480 0x7f6754199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f6759813700 1 --2- 172.21.15.19:0/3168884744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6754102ac0 0x7f6754199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f675a014700 1 -- 172.21.15.19:0/3168884744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6754102ac0 msgr2=0x7f6754199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f675a014700 1 --2- 172.21.15.19:0/3168884744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6754102ac0 0x7f6754199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f675a014700 1 -- 172.21.15.19:0/3168884744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6754103480 msgr2=0x7f6754199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:09.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f675a014700 1 --2- 172.21.15.19:0/3168884744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6754103480 0x7f6754199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f675a014700 1 -- 172.21.15.19:0/3168884744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675419e230 con 0x7f6754108ab0 2024-04-25T12:06:09.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f675a014700 1 --2- 172.21.15.19:0/3168884744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6754108ab0 0x7f675419daf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f674c00ada0 tx=0x7f674c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:09.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f674affd700 1 -- 172.21.15.19:0/3168884744 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c017020 con 0x7f6754108ab0 2024-04-25T12:06:09.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f675ba77700 1 -- 172.21.15.19:0/3168884744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675419e520 con 0x7f6754108ab0 2024-04-25T12:06:09.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f674affd700 1 -- 172.21.15.19:0/3168884744 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f674c00ce50 con 0x7f6754108ab0 2024-04-25T12:06:09.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f674affd700 1 -- 172.21.15.19:0/3168884744 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c005360 con 0x7f6754108ab0 2024-04-25T12:06:09.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.414+0000 7f675ba77700 1 -- 172.21.15.19:0/3168884744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67541a5da0 con 0x7f6754108ab0 2024-04-25T12:06:09.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.416+0000 7f675ba77700 1 -- 172.21.15.19:0/3168884744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6754069730 con 0x7f6754108ab0 2024-04-25T12:06:09.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.417+0000 7f674affd700 1 -- 172.21.15.19:0/3168884744 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f674c00c980 con 0x7f6754108ab0 2024-04-25T12:06:09.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.418+0000 7f674affd700 1 --2- 172.21.15.19:0/3168884744 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f67400a3cd0 0x7f67400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:09.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.418+0000 7f6759813700 1 --2- 172.21.15.19:0/3168884744 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f67400a3cd0 0x7f67400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:09.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.418+0000 7f674affd700 1 -- 172.21.15.19:0/3168884744 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f674c0cfca0 con 0x7f6754108ab0 2024-04-25T12:06:09.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.418+0000 7f6759813700 1 --2- 172.21.15.19:0/3168884744 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f67400a3cd0 0x7f67400a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6744005fd0 tx=0x7f6744005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:09.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.422+0000 7f674affd700 1 -- 172.21.15.19:0/3168884744 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f674c021020 con 0x7f6754108ab0 2024-04-25T12:06:09.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.633+0000 7f675ba77700 1 -- 172.21.15.19:0/3168884744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f675402df80 con 0x7f6754108ab0 2024-04-25T12:06:09.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.634+0000 7f674affd700 1 -- 172.21.15.19:0/3168884744 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f674c00cc70 con 0x7f6754108ab0 2024-04-25T12:06:09.633 INFO:teuthology.orchestra.run.smithi019.stdout:34359738451 2024-04-25T12:06:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.636+0000 7f675ba77700 1 -- 172.21.15.19:0/3168884744 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f67400a3cd0 msgr2=0x7f67400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.636+0000 7f675ba77700 1 --2- 172.21.15.19:0/3168884744 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f67400a3cd0 0x7f67400a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6744005fd0 tx=0x7f6744005db0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.636+0000 7f675ba77700 1 -- 172.21.15.19:0/3168884744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6754108ab0 msgr2=0x7f675419daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.636+0000 7f675ba77700 1 --2- 172.21.15.19:0/3168884744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6754108ab0 0x7f675419daf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f674c00ada0 tx=0x7f674c00c330 comp rx=0 tx=0).stop 2024-04-25T12:06:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.636+0000 7f675ba77700 1 -- 172.21.15.19:0/3168884744 shutdown_connections 2024-04-25T12:06:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.636+0000 7f675ba77700 1 --2- 172.21.15.19:0/3168884744 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f67400a3cd0 0x7f67400a6190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.636+0000 7f675ba77700 1 --2- 172.21.15.19:0/3168884744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6754108ab0 0x7f675419daf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.636+0000 7f675ba77700 1 --2- 172.21.15.19:0/3168884744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6754102ac0 0x7f6754199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.636+0000 7f675ba77700 1 --2- 172.21.15.19:0/3168884744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6754103480 0x7f6754199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.636+0000 7f675ba77700 1 -- 172.21.15.19:0/3168884744 >> 172.21.15.19:0/3168884744 conn(0x7f6754079bf0 msgr2=0x7f6754100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.637+0000 7f675ba77700 1 -- 172.21.15.19:0/3168884744 shutdown_connections 2024-04-25T12:06:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.637+0000 7f675ba77700 1 -- 172.21.15.19:0/3168884744 wait complete. 2024-04-25T12:06:09.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:09 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3607831696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T12:06:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.310+0000 7fc3b584a700 1 -- 172.21.15.19:0/4136103001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3b0107990 msgr2=0x7fc3b0107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:09.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.310+0000 7fc3b584a700 1 --2- 172.21.15.19:0/4136103001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3b0107990 0x7fc3b0107d70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc3a00098f0 tx=0x7fc3a0009c00 comp rx=0 tx=0).stop 2024-04-25T12:06:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.310+0000 7fc3b584a700 1 -- 172.21.15.19:0/4136103001 shutdown_connections 2024-04-25T12:06:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.310+0000 7fc3b584a700 1 --2- 172.21.15.19:0/4136103001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b0102300 0x7fc3b010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.310+0000 7fc3b584a700 1 --2- 172.21.15.19:0/4136103001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3b0107990 0x7fc3b0107d70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.310+0000 7fc3b584a700 1 --2- 172.21.15.19:0/4136103001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3b0101940 0x7fc3b0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.310+0000 7fc3b584a700 1 -- 172.21.15.19:0/4136103001 >> 172.21.15.19:0/4136103001 conn(0x7fc3b00788f0 msgr2=0x7fc3b0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.310+0000 7fc3b584a700 1 -- 172.21.15.19:0/4136103001 shutdown_connections 2024-04-25T12:06:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.310+0000 7fc3b584a700 1 -- 172.21.15.19:0/4136103001 wait complete. 2024-04-25T12:06:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.311+0000 7fc3b584a700 1 Processor -- start 2024-04-25T12:06:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.311+0000 7fc3b584a700 1 -- start start 2024-04-25T12:06:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.311+0000 7fc3b584a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3b0101940 0x7fc3b0199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.311+0000 7fc3b584a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3b0102300 0x7fc3b0199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.311+0000 7fc3b584a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b0107990 0x7fc3b019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.311+0000 7fc3b584a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b0111c40 con 0x7fc3b0101940 2024-04-25T12:06:09.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.311+0000 7fc3b584a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b0111ac0 con 0x7fc3b0102300 2024-04-25T12:06:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.311+0000 7fc3b584a700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b0111dc0 con 0x7fc3b0107990 2024-04-25T12:06:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.312+0000 7fc3aeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3b0101940 0x7fc3b0199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.312+0000 7fc3aeffd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3b0101940 0x7fc3b0199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57784/0 (socket says 172.21.15.19:57784) 2024-04-25T12:06:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.312+0000 7fc3aeffd700 1 -- 172.21.15.19:0/3607831696 learned_addr learned my addr 172.21.15.19:0/3607831696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.312+0000 7fc3ae7fc700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3b0102300 0x7fc3b0199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.312+0000 7fc3af7fe700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b0107990 0x7fc3b019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.312+0000 7fc3aeffd700 1 -- 172.21.15.19:0/3607831696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b0107990 msgr2=0x7fc3b019db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.312+0000 7fc3aeffd700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b0107990 0x7fc3b019db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.312+0000 7fc3aeffd700 1 -- 172.21.15.19:0/3607831696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3b0102300 msgr2=0x7fc3b0199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.312+0000 7fc3aeffd700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3b0102300 0x7fc3b0199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.312+0000 7fc3aeffd700 1 -- 172.21.15.19:0/3607831696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b019e250 con 0x7fc3b0101940 2024-04-25T12:06:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.312+0000 7fc3af7fe700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b0107990 0x7fc3b019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:06:09.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.312+0000 7fc3aeffd700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3b0101940 0x7fc3b0199240 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009230 tx=0x7fc3a0010fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:09.746 INFO:teuthology.orchestra.run.smithi019.stdout:343597383706 2024-04-25T12:06:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.313+0000 7fc3b4848700 1 -- 172.21.15.19:0/3607831696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0027070 con 0x7fc3b0101940 2024-04-25T12:06:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.313+0000 7fc3b4848700 1 -- 172.21.15.19:0/3607831696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc3a000b560 con 0x7fc3b0101940 2024-04-25T12:06:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.313+0000 7fc3b4848700 1 -- 172.21.15.19:0/3607831696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0004b30 con 0x7fc3b0101940 2024-04-25T12:06:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.313+0000 7fc3b584a700 1 -- 172.21.15.19:0/3607831696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b019e4e0 con 0x7fc3b0101940 2024-04-25T12:06:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.313+0000 7fc3b584a700 1 -- 172.21.15.19:0/3607831696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b01a5dc0 con 0x7fc3b0101940 2024-04-25T12:06:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.314+0000 7fc3b584a700 1 -- 172.21.15.19:0/3607831696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b0103060 con 0x7fc3b0101940 2024-04-25T12:06:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.315+0000 7fc3b4848700 1 -- 172.21.15.19:0/3607831696 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc3a002a070 con 0x7fc3b0101940 2024-04-25T12:06:09.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.316+0000 7fc3b4848700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc3980a3d50 0x7fc3980a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.316+0000 7fc3ae7fc700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc3980a3d50 0x7fc3980a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.316+0000 7fc3b4848700 1 -- 172.21.15.19:0/3607831696 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc3a00d1550 con 0x7fc3b0101940 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.317+0000 7fc3ae7fc700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc3980a3d50 0x7fc3980a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0078e80 tx=0x7fc3a4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.321+0000 7fc3b4848700 1 -- 172.21.15.19:0/3607831696 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3a0031070 con 0x7fc3b0101940 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.533+0000 7fc3b584a700 1 -- 172.21.15.19:0/3607831696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fc3b0069730 con 0x7fc3b0101940 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.533+0000 7fc3b4848700 1 -- 172.21.15.19:0/3607831696 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc3a0013030 con 0x7fc3b0101940 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.536+0000 7fc3b584a700 1 -- 172.21.15.19:0/3607831696 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc3980a3d50 msgr2=0x7fc3980a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.536+0000 7fc3b584a700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc3980a3d50 0x7fc3980a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0078e80 tx=0x7fc3a4009310 comp rx=0 tx=0).stop 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.536+0000 7fc3b584a700 1 -- 172.21.15.19:0/3607831696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3b0101940 msgr2=0x7fc3b0199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.536+0000 7fc3b584a700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3b0101940 0x7fc3b0199240 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009230 tx=0x7fc3a0010fb0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.536+0000 7fc3b584a700 1 -- 172.21.15.19:0/3607831696 shutdown_connections 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.536+0000 7fc3b584a700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc3980a3d50 0x7fc3980a6210 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.536+0000 7fc3b584a700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3b0101940 0x7fc3b0199240 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.536+0000 7fc3b584a700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3b0107990 0x7fc3b019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.536+0000 7fc3b584a700 1 --2- 172.21.15.19:0/3607831696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3b0102300 0x7fc3b0199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:09.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.536+0000 7fc3b584a700 1 -- 172.21.15.19:0/3607831696 >> 172.21.15.19:0/3607831696 conn(0x7fc3b00788f0 msgr2=0x7fc3b006a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.536+0000 7fc3b584a700 1 -- 172.21.15.19:0/3607831696 shutdown_connections 2024-04-25T12:06:09.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:09.536+0000 7fc3b584a700 1 -- 172.21.15.19:0/3607831696 wait complete. 2024-04-25T12:06:09.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:09 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3607831696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T12:06:09.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:09 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3607831696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T12:06:10.402 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574925 for osd.1 2024-04-25T12:06:10.403 DEBUG:teuthology.parallel:result is None 2024-04-25T12:06:10.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:10 smithi019 ceph-mon[29213]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5 op/s; 107 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T12:06:10.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:10 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3168884744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T12:06:10.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:10 smithi169 ceph-mon[32710]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5 op/s; 107 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T12:06:10.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:10 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3168884744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T12:06:10.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:10 smithi150 ceph-mon[32250]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5 op/s; 107 B/s, 0 keys/s, 1 objects/s recovering 2024-04-25T12:06:10.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:10 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3168884744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T12:06:11.771 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:11 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3113131358' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T12:06:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.775+0000 7fe15f7cf700 1 -- 172.21.15.19:0/1249783207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe158107990 msgr2=0x7fe158107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.775+0000 7fe15f7cf700 1 --2- 172.21.15.19:0/1249783207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe158107990 0x7fe158107d70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe154009990 tx=0x7fe154009ca0 comp rx=0 tx=0).stop 2024-04-25T12:06:11.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.775+0000 7fe15f7cf700 1 -- 172.21.15.19:0/1249783207 shutdown_connections 2024-04-25T12:06:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.775+0000 7fe15f7cf700 1 --2- 172.21.15.19:0/1249783207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe158107990 0x7fe158107d70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.775+0000 7fe15f7cf700 1 --2- 172.21.15.19:0/1249783207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe158102300 0x7fe15810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.775+0000 7fe15f7cf700 1 --2- 172.21.15.19:0/1249783207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe158101940 0x7fe158101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.775+0000 7fe15f7cf700 1 -- 172.21.15.19:0/1249783207 >> 172.21.15.19:0/1249783207 conn(0x7fe1580788f0 msgr2=0x7fe158078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.775+0000 7fe15f7cf700 1 -- 172.21.15.19:0/1249783207 shutdown_connections 2024-04-25T12:06:11.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.775+0000 7fe15f7cf700 1 -- 172.21.15.19:0/1249783207 wait complete. 2024-04-25T12:06:11.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.776+0000 7fe15f7cf700 1 Processor -- start 2024-04-25T12:06:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.776+0000 7fe15f7cf700 1 -- start start 2024-04-25T12:06:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.776+0000 7fe15f7cf700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe158101940 0x7fe1581980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.776+0000 7fe15f7cf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe158102300 0x7fe158198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.776+0000 7fe15f7cf700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe158107990 0x7fe15819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.776+0000 7fe15f7cf700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe158111ce0 con 0x7fe158107990 2024-04-25T12:06:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.776+0000 7fe15f7cf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe158111b60 con 0x7fe158102300 2024-04-25T12:06:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.776+0000 7fe15f7cf700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe158111e60 con 0x7fe158101940 2024-04-25T12:06:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.776+0000 7fe15dd6c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe158107990 0x7fe15819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.776+0000 7fe15dd6c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe158107990 0x7fe15819c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50394/0 (socket says 172.21.15.19:50394) 2024-04-25T12:06:11.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.776+0000 7fe15dd6c700 1 -- 172.21.15.19:0/2765886501 learned_addr learned my addr 172.21.15.19:0/2765886501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.776+0000 7fe15d56b700 1 --2- 172.21.15.19:0/2765886501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe158101940 0x7fe1581980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.776+0000 7fe15cd6a700 1 --2- 172.21.15.19:0/2765886501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe158102300 0x7fe158198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.777+0000 7fe15dd6c700 1 -- 172.21.15.19:0/2765886501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe158101940 msgr2=0x7fe1581980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.777+0000 7fe15dd6c700 1 --2- 172.21.15.19:0/2765886501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe158101940 0x7fe1581980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.777+0000 7fe15dd6c700 1 -- 172.21.15.19:0/2765886501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe158102300 msgr2=0x7fe158198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.777+0000 7fe15dd6c700 1 --2- 172.21.15.19:0/2765886501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe158102300 0x7fe158198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.777+0000 7fe15dd6c700 1 -- 172.21.15.19:0/2765886501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15819d100 con 0x7fe158107990 2024-04-25T12:06:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.777+0000 7fe15dd6c700 1 --2- 172.21.15.19:0/2765886501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe158107990 0x7fe15819c9c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe150007d70 tx=0x7fe15000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.777+0000 7fe14e7fc700 1 -- 172.21.15.19:0/2765886501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150016020 con 0x7fe158107990 2024-04-25T12:06:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.777+0000 7fe14e7fc700 1 -- 172.21.15.19:0/2765886501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe15000ac90 con 0x7fe158107990 2024-04-25T12:06:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.777+0000 7fe15f7cf700 1 -- 172.21.15.19:0/2765886501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15819d3f0 con 0x7fe158107990 2024-04-25T12:06:11.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.777+0000 7fe14e7fc700 1 -- 172.21.15.19:0/2765886501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1500115e0 con 0x7fe158107990 2024-04-25T12:06:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.777+0000 7fe15f7cf700 1 -- 172.21.15.19:0/2765886501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1581a4c70 con 0x7fe158107990 2024-04-25T12:06:11.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.778+0000 7fe15f7cf700 1 -- 172.21.15.19:0/2765886501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe158109eb0 con 0x7fe158107990 2024-04-25T12:06:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.780+0000 7fe14e7fc700 1 -- 172.21.15.19:0/2765886501 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe150019020 con 0x7fe158107990 2024-04-25T12:06:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.781+0000 7fe14e7fc700 1 --2- 172.21.15.19:0/2765886501 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe1440a3cd0 0x7fe1440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.781+0000 7fe14e7fc700 1 -- 172.21.15.19:0/2765886501 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fe1500d0820 con 0x7fe158107990 2024-04-25T12:06:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.781+0000 7fe15d56b700 1 --2- 172.21.15.19:0/2765886501 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe1440a3cd0 0x7fe1440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:11.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.782+0000 7fe15d56b700 1 --2- 172.21.15.19:0/2765886501 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe1440a3cd0 0x7fe1440a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe154009750 tx=0x7fe154005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:11.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.784+0000 7fe14e7fc700 1 -- 172.21.15.19:0/2765886501 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe150020020 con 0x7fe158107990 2024-04-25T12:06:11.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:11 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3113131358' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T12:06:11.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:11 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3113131358' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T12:06:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.996+0000 7fe15f7cf700 1 -- 172.21.15.19:0/2765886501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fe158109890 con 0x7fe158107990 2024-04-25T12:06:11.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.996+0000 7fe14e7fc700 1 -- 172.21.15.19:0/2765886501 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe15009d190 con 0x7fe158107990 2024-04-25T12:06:11.995 INFO:teuthology.orchestra.run.smithi019.stdout:309237645343 2024-04-25T12:06:11.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.998+0000 7fe15f7cf700 1 -- 172.21.15.19:0/2765886501 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe1440a3cd0 msgr2=0x7fe1440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.998+0000 7fe15f7cf700 1 --2- 172.21.15.19:0/2765886501 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe1440a3cd0 0x7fe1440a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe154009750 tx=0x7fe154005fd0 comp rx=0 tx=0).stop 2024-04-25T12:06:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.998+0000 7fe15f7cf700 1 -- 172.21.15.19:0/2765886501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe158107990 msgr2=0x7fe15819c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.998+0000 7fe15f7cf700 1 --2- 172.21.15.19:0/2765886501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe158107990 0x7fe15819c9c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe150007d70 tx=0x7fe15000a340 comp rx=0 tx=0).stop 2024-04-25T12:06:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.998+0000 7fe15f7cf700 1 -- 172.21.15.19:0/2765886501 shutdown_connections 2024-04-25T12:06:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.998+0000 7fe15f7cf700 1 --2- 172.21.15.19:0/2765886501 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe1440a3cd0 0x7fe1440a6190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.998+0000 7fe15f7cf700 1 --2- 172.21.15.19:0/2765886501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe158107990 0x7fe15819c9c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.998+0000 7fe15f7cf700 1 --2- 172.21.15.19:0/2765886501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe158101940 0x7fe1581980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.998+0000 7fe15f7cf700 1 --2- 172.21.15.19:0/2765886501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe158102300 0x7fe158198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.998+0000 7fe15f7cf700 1 -- 172.21.15.19:0/2765886501 >> 172.21.15.19:0/2765886501 conn(0x7fe1580788f0 msgr2=0x7fe158100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:11.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.998+0000 7fe15f7cf700 1 -- 172.21.15.19:0/2765886501 shutdown_connections 2024-04-25T12:06:11.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.999+0000 7fe15f7cf700 1 -- 172.21.15.19:0/2765886501 wait complete. 2024-04-25T12:06:12.209 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738451 for osd.0 2024-04-25T12:06:12.209 DEBUG:teuthology.parallel:result is None 2024-04-25T12:06:12.811 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:12 smithi019 ceph-mon[29213]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5 op/s; 51 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T12:06:12.812 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:12 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2765886501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T12:06:12.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.811+0000 7fe76a4e4700 1 -- 172.21.15.19:0/647453777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764101940 msgr2=0x7fe764101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:12.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.811+0000 7fe76a4e4700 1 --2- 172.21.15.19:0/647453777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764101940 0x7fe764101dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe758009990 tx=0x7fe758009ca0 comp rx=0 tx=0).stop 2024-04-25T12:06:12.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.811+0000 7fe76a4e4700 1 -- 172.21.15.19:0/647453777 shutdown_connections 2024-04-25T12:06:12.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.811+0000 7fe76a4e4700 1 --2- 172.21.15.19:0/647453777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe764102300 0x7fe76410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:12.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.811+0000 7fe76a4e4700 1 --2- 172.21.15.19:0/647453777 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe764107990 0x7fe764107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:12.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.811+0000 7fe76a4e4700 1 --2- 172.21.15.19:0/647453777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764101940 0x7fe764101dc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:12.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.811+0000 7fe76a4e4700 1 -- 172.21.15.19:0/647453777 >> 172.21.15.19:0/647453777 conn(0x7fe7640788f0 msgr2=0x7fe764078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:12.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.811+0000 7fe76a4e4700 1 -- 172.21.15.19:0/647453777 shutdown_connections 2024-04-25T12:06:12.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.811+0000 7fe76a4e4700 1 -- 172.21.15.19:0/647453777 wait complete. 2024-04-25T12:06:12.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.812+0000 7fe76a4e4700 1 Processor -- start 2024-04-25T12:06:12.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.812+0000 7fe76a4e4700 1 -- start start 2024-04-25T12:06:12.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.812+0000 7fe76a4e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764101940 0x7fe76419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:12.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.812+0000 7fe76a4e4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe764102300 0x7fe76419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:12.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.812+0000 7fe76a4e4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe764107990 0x7fe76419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:12.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.812+0000 7fe76a4e4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe764111c70 con 0x7fe764101940 2024-04-25T12:06:12.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.812+0000 7fe76a4e4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe764111af0 con 0x7fe764107990 2024-04-25T12:06:12.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.812+0000 7fe76a4e4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe764111df0 con 0x7fe764102300 2024-04-25T12:06:12.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.812+0000 7fe763fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764101940 0x7fe76419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:12.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.812+0000 7fe763fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764101940 0x7fe76419b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50422/0 (socket says 172.21.15.19:50422) 2024-04-25T12:06:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.812+0000 7fe763fff700 1 -- 172.21.15.19:0/2717137087 learned_addr learned my addr 172.21.15.19:0/2717137087 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.812+0000 7fe7637fe700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe764102300 0x7fe76419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.812+0000 7fe768a81700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe764107990 0x7fe76419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.812+0000 7fe763fff700 1 -- 172.21.15.19:0/2717137087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe764102300 msgr2=0x7fe76419b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.813+0000 7fe763fff700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe764102300 0x7fe76419b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.813+0000 7fe763fff700 1 -- 172.21.15.19:0/2717137087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe764107990 msgr2=0x7fe76419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.813+0000 7fe763fff700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe764107990 0x7fe76419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.813+0000 7fe763fff700 1 -- 172.21.15.19:0/2717137087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7641a0400 con 0x7fe764101940 2024-04-25T12:06:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.813+0000 7fe768a81700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe764107990 0x7fe76419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.813+0000 7fe763fff700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764101940 0x7fe76419b3f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe75400d950 tx=0x7fe75400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.813+0000 7fe7617fa700 1 -- 172.21.15.19:0/2717137087 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754016070 con 0x7fe764101940 2024-04-25T12:06:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.813+0000 7fe7617fa700 1 -- 172.21.15.19:0/2717137087 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe754009b40 con 0x7fe764101940 2024-04-25T12:06:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.813+0000 7fe76a4e4700 1 -- 172.21.15.19:0/2717137087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7641a06f0 con 0x7fe764101940 2024-04-25T12:06:12.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.813+0000 7fe7617fa700 1 -- 172.21.15.19:0/2717137087 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75400f620 con 0x7fe764101940 2024-04-25T12:06:12.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.813+0000 7fe76a4e4700 1 -- 172.21.15.19:0/2717137087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7641a7f70 con 0x7fe764101940 2024-04-25T12:06:12.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.814+0000 7fe76a4e4700 1 -- 172.21.15.19:0/2717137087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe764069730 con 0x7fe764101940 2024-04-25T12:06:12.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.816+0000 7fe7617fa700 1 -- 172.21.15.19:0/2717137087 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe754019070 con 0x7fe764101940 2024-04-25T12:06:12.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.816+0000 7fe7617fa700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe74c0a3d50 0x7fe74c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:12.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.816+0000 7fe7637fe700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe74c0a3d50 0x7fe74c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:12.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.816+0000 7fe7617fa700 1 -- 172.21.15.19:0/2717137087 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fe7540d0100 con 0x7fe764101940 2024-04-25T12:06:12.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.817+0000 7fe7637fe700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe74c0a3d50 0x7fe74c0a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe758006010 tx=0x7fe75800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:12.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:12.821+0000 7fe7617fa700 1 -- 172.21.15.19:0/2717137087 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe75409c960 con 0x7fe764101940 2024-04-25T12:06:12.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:12 smithi169 ceph-mon[32710]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5 op/s; 51 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T12:06:12.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:12 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2765886501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T12:06:12.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:12 smithi150 ceph-mon[32250]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5 op/s; 51 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T12:06:12.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:12 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2765886501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T12:06:13.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.033+0000 7fe76a4e4700 1 -- 172.21.15.19:0/2717137087 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe7641a8200 con 0x7fe764101940 2024-04-25T12:06:13.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.033+0000 7fe7617fa700 1 -- 172.21.15.19:0/2717137087 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe75409c260 con 0x7fe764101940 2024-04-25T12:06:13.032 INFO:teuthology.orchestra.run.smithi019.stdout:219043332144 2024-04-25T12:06:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.036+0000 7fe76a4e4700 1 -- 172.21.15.19:0/2717137087 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe74c0a3d50 msgr2=0x7fe74c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.036+0000 7fe76a4e4700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe74c0a3d50 0x7fe74c0a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe758006010 tx=0x7fe75800b540 comp rx=0 tx=0).stop 2024-04-25T12:06:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.036+0000 7fe76a4e4700 1 -- 172.21.15.19:0/2717137087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764101940 msgr2=0x7fe76419b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.036+0000 7fe76a4e4700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764101940 0x7fe76419b3f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe75400d950 tx=0x7fe75400dd10 comp rx=0 tx=0).stop 2024-04-25T12:06:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.036+0000 7fe76a4e4700 1 -- 172.21.15.19:0/2717137087 shutdown_connections 2024-04-25T12:06:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.036+0000 7fe76a4e4700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe74c0a3d50 0x7fe74c0a6210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.036+0000 7fe76a4e4700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe764101940 0x7fe76419b3f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.036+0000 7fe76a4e4700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe764107990 0x7fe76419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.036+0000 7fe76a4e4700 1 --2- 172.21.15.19:0/2717137087 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe764102300 0x7fe76419b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.036+0000 7fe76a4e4700 1 -- 172.21.15.19:0/2717137087 >> 172.21.15.19:0/2717137087 conn(0x7fe7640788f0 msgr2=0x7fe76406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.036+0000 7fe76a4e4700 1 -- 172.21.15.19:0/2717137087 shutdown_connections 2024-04-25T12:06:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.036+0000 7fe76a4e4700 1 -- 172.21.15.19:0/2717137087 wait complete. 2024-04-25T12:06:13.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:13 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2717137087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T12:06:13.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:13 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3290855658' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T12:06:13.869 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332140 got 219043332144 for osd.7 2024-04-25T12:06:13.869 DEBUG:teuthology.parallel:result is None 2024-04-25T12:06:13.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:13 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2717137087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T12:06:13.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:13 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3290855658' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T12:06:13.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:13 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2717137087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T12:06:13.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:13 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3290855658' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T12:06:14.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.290+0000 7f48a34c1700 1 -- 172.21.15.19:0/1935187142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f489c1079a0 msgr2=0x7f489c06b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.290+0000 7f48a34c1700 1 --2- 172.21.15.19:0/1935187142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f489c1079a0 0x7f489c06b1f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4898009990 tx=0x7f4898009ca0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.290+0000 7f48a34c1700 1 -- 172.21.15.19:0/1935187142 shutdown_connections 2024-04-25T12:06:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.290+0000 7f48a34c1700 1 --2- 172.21.15.19:0/1935187142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f489c1079a0 0x7f489c06b1f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.290+0000 7f48a34c1700 1 --2- 172.21.15.19:0/1935187142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f489c1054b0 0x7f489c105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.290+0000 7f48a34c1700 1 --2- 172.21.15.19:0/1935187142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f489c06b730 0x7f489c06bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.290+0000 7f48a34c1700 1 -- 172.21.15.19:0/1935187142 >> 172.21.15.19:0/1935187142 conn(0x7f489c0789e0 msgr2=0x7f489c078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.290+0000 7f48a34c1700 1 -- 172.21.15.19:0/1935187142 shutdown_connections 2024-04-25T12:06:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.290+0000 7f48a34c1700 1 -- 172.21.15.19:0/1935187142 wait complete. 2024-04-25T12:06:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a34c1700 1 Processor -- start 2024-04-25T12:06:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a34c1700 1 -- start start 2024-04-25T12:06:14.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a34c1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f489c06b730 0x7f489c1a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:14.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a34c1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f489c1054b0 0x7f489c1a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:14.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a34c1700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f489c1079a0 0x7f489c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:14.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a34c1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f489c119380 con 0x7f489c06b730 2024-04-25T12:06:14.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a34c1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f489c119200 con 0x7f489c1054b0 2024-04-25T12:06:14.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a34c1700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f489c119500 con 0x7f489c1079a0 2024-04-25T12:06:14.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a125d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f489c06b730 0x7f489c1a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:14.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a125d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f489c06b730 0x7f489c1a5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50460/0 (socket says 172.21.15.19:50460) 2024-04-25T12:06:14.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a125d700 1 -- 172.21.15.19:0/917802032 learned_addr learned my addr 172.21.15.19:0/917802032 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:14.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a0a5c700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f489c1054b0 0x7f489c1a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:14.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a1a5e700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f489c1079a0 0x7f489c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:14.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a125d700 1 -- 172.21.15.19:0/917802032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f489c1079a0 msgr2=0x7f489c19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a125d700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f489c1079a0 0x7f489c19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a125d700 1 -- 172.21.15.19:0/917802032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f489c1054b0 msgr2=0x7f489c1a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a125d700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f489c1054b0 0x7f489c1a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a125d700 1 -- 172.21.15.19:0/917802032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c19fd80 con 0x7f489c06b730 2024-04-25T12:06:14.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.291+0000 7f48a0a5c700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f489c1054b0 0x7f489c1a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:14.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.292+0000 7f48a125d700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f489c06b730 0x7f489c1a5410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f48980047b0 tx=0x7f4898009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:14.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.292+0000 7f48927fc700 1 -- 172.21.15.19:0/917802032 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898027070 con 0x7f489c06b730 2024-04-25T12:06:14.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.292+0000 7f48927fc700 1 -- 172.21.15.19:0/917802032 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4898004100 con 0x7f489c06b730 2024-04-25T12:06:14.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.292+0000 7f48927fc700 1 -- 172.21.15.19:0/917802032 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489800f460 con 0x7f489c06b730 2024-04-25T12:06:14.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.292+0000 7f48a34c1700 1 -- 172.21.15.19:0/917802032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c1a0010 con 0x7f489c06b730 2024-04-25T12:06:14.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.292+0000 7f48a34c1700 1 -- 172.21.15.19:0/917802032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c1ac2e0 con 0x7f489c06b730 2024-04-25T12:06:14.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.293+0000 7f48a34c1700 1 -- 172.21.15.19:0/917802032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f489c101f90 con 0x7f489c06b730 2024-04-25T12:06:14.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.294+0000 7f48927fc700 1 -- 172.21.15.19:0/917802032 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f489802a070 con 0x7f489c06b730 2024-04-25T12:06:14.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.295+0000 7f48927fc700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f48880a3cd0 0x7f48880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:14.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.295+0000 7f48a0a5c700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f48880a3cd0 0x7f48880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:14.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.295+0000 7f48927fc700 1 -- 172.21.15.19:0/917802032 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f48980d1250 con 0x7f489c06b730 2024-04-25T12:06:14.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.295+0000 7f48a0a5c700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f48880a3cd0 0x7f48880a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f488c005950 tx=0x7f488c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:14.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.299+0000 7f48927fc700 1 -- 172.21.15.19:0/917802032 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4898013070 con 0x7f489c06b730 2024-04-25T12:06:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.281+0000 7fd21389c700 1 -- 172.21.15.19:0/1454869285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20c103450 msgr2=0x7fd20c10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.281+0000 7fd21389c700 1 --2- 172.21.15.19:0/1454869285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20c103450 0x7fd20c10f8b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd204009990 tx=0x7fd204009ca0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.281+0000 7fd21389c700 1 -- 172.21.15.19:0/1454869285 shutdown_connections 2024-04-25T12:06:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.281+0000 7fd21389c700 1 --2- 172.21.15.19:0/1454869285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20c103450 0x7fd20c10f8b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.281+0000 7fd21389c700 1 --2- 172.21.15.19:0/1454869285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd20c108a50 0x7fd20c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.281+0000 7fd21389c700 1 --2- 172.21.15.19:0/1454869285 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20c102a90 0x7fd20c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.281+0000 7fd21389c700 1 -- 172.21.15.19:0/1454869285 >> 172.21.15.19:0/1454869285 conn(0x7fd20c079ba0 msgr2=0x7fd20c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.281+0000 7fd21389c700 1 -- 172.21.15.19:0/1454869285 shutdown_connections 2024-04-25T12:06:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.281+0000 7fd21389c700 1 -- 172.21.15.19:0/1454869285 wait complete. 2024-04-25T12:06:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.281+0000 7fd21389c700 1 Processor -- start 2024-04-25T12:06:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd21389c700 1 -- start start 2024-04-25T12:06:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd21389c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd20c102a90 0x7fd20c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd21389c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20c103450 0x7fd20c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd21389c700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20c108a50 0x7fd20c19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd21389c700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c112d40 con 0x7fd20c108a50 2024-04-25T12:06:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd21389c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c112bc0 con 0x7fd20c102a90 2024-04-25T12:06:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd21389c700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c112ec0 con 0x7fd20c103450 2024-04-25T12:06:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd211e39700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20c108a50 0x7fd20c19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd211e39700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20c108a50 0x7fd20c19da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50438/0 (socket says 172.21.15.19:50438) 2024-04-25T12:06:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd211e39700 1 -- 172.21.15.19:0/3290855658 learned_addr learned my addr 172.21.15.19:0/3290855658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd211638700 1 --2- 172.21.15.19:0/3290855658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd20c102a90 0x7fd20c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd211e39700 1 -- 172.21.15.19:0/3290855658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20c103450 msgr2=0x7fd20c1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd211e39700 1 --2- 172.21.15.19:0/3290855658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20c103450 0x7fd20c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd211e39700 1 -- 172.21.15.19:0/3290855658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd20c102a90 msgr2=0x7fd20c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.413 INFO:teuthology.orchestra.run.smithi019.stdout:141733920831 2024-04-25T12:06:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd211e39700 1 --2- 172.21.15.19:0/3290855658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd20c102a90 0x7fd20c199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd211e39700 1 -- 172.21.15.19:0/3290855658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20c19e180 con 0x7fd20c108a50 2024-04-25T12:06:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd211638700 1 --2- 172.21.15.19:0/3290855658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd20c102a90 0x7fd20c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd211e39700 1 --2- 172.21.15.19:0/3290855658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20c108a50 0x7fd20c19da40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd204005ea0 tx=0x7fd204009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd20a7fc700 1 -- 172.21.15.19:0/3290855658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204027070 con 0x7fd20c108a50 2024-04-25T12:06:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd20a7fc700 1 -- 172.21.15.19:0/3290855658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd204003fb0 con 0x7fd20c108a50 2024-04-25T12:06:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd21389c700 1 -- 172.21.15.19:0/3290855658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20c19e410 con 0x7fd20c108a50 2024-04-25T12:06:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.282+0000 7fd20a7fc700 1 -- 172.21.15.19:0/3290855658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204022d00 con 0x7fd20c108a50 2024-04-25T12:06:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.283+0000 7fd21389c700 1 -- 172.21.15.19:0/3290855658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd20c1a5cf0 con 0x7fd20c108a50 2024-04-25T12:06:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.283+0000 7fd21389c700 1 -- 172.21.15.19:0/3290855658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd20c069730 con 0x7fd20c108a50 2024-04-25T12:06:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.285+0000 7fd20a7fc700 1 -- 172.21.15.19:0/3290855658 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd20402a070 con 0x7fd20c108a50 2024-04-25T12:06:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.286+0000 7fd20a7fc700 1 --2- 172.21.15.19:0/3290855658 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd1f40a3d50 0x7fd1f40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.286+0000 7fd211638700 1 --2- 172.21.15.19:0/3290855658 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd1f40a3d50 0x7fd1f40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.286+0000 7fd20a7fc700 1 -- 172.21.15.19:0/3290855658 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd2040d1930 con 0x7fd20c108a50 2024-04-25T12:06:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.286+0000 7fd211638700 1 --2- 172.21.15.19:0/3290855658 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd1f40a3d50 0x7fd1f40a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009d50 tx=0x7fd1fc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.290+0000 7fd20a7fc700 1 -- 172.21.15.19:0/3290855658 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd204031020 con 0x7fd20c108a50 2024-04-25T12:06:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.500+0000 7fd21389c700 1 -- 172.21.15.19:0/3290855658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd20c02df80 con 0x7fd20c108a50 2024-04-25T12:06:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.501+0000 7fd20a7fc700 1 -- 172.21.15.19:0/3290855658 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd204013070 con 0x7fd20c108a50 2024-04-25T12:06:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.503+0000 7fd21389c700 1 -- 172.21.15.19:0/3290855658 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd1f40a3d50 msgr2=0x7fd1f40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.503+0000 7fd21389c700 1 --2- 172.21.15.19:0/3290855658 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd1f40a3d50 0x7fd1f40a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009d50 tx=0x7fd1fc009040 comp rx=0 tx=0).stop 2024-04-25T12:06:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.503+0000 7fd21389c700 1 -- 172.21.15.19:0/3290855658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20c108a50 msgr2=0x7fd20c19da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.503+0000 7fd21389c700 1 --2- 172.21.15.19:0/3290855658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20c108a50 0x7fd20c19da40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd204005ea0 tx=0x7fd204009ee0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.503+0000 7fd21389c700 1 -- 172.21.15.19:0/3290855658 shutdown_connections 2024-04-25T12:06:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.503+0000 7fd21389c700 1 --2- 172.21.15.19:0/3290855658 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd1f40a3d50 0x7fd1f40a6210 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.503+0000 7fd21389c700 1 --2- 172.21.15.19:0/3290855658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd20c108a50 0x7fd20c19da40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.503+0000 7fd21389c700 1 --2- 172.21.15.19:0/3290855658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd20c102a90 0x7fd20c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.503+0000 7fd21389c700 1 --2- 172.21.15.19:0/3290855658 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd20c103450 0x7fd20c1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.503+0000 7fd21389c700 1 -- 172.21.15.19:0/3290855658 >> 172.21.15.19:0/3290855658 conn(0x7fd20c079ba0 msgr2=0x7fd20c1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.503+0000 7fd21389c700 1 -- 172.21.15.19:0/3290855658 shutdown_connections 2024-04-25T12:06:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:13.503+0000 7fd21389c700 1 -- 172.21.15.19:0/3290855658 wait complete. 2024-04-25T12:06:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.509+0000 7f48a34c1700 1 -- 172.21.15.19:0/917802032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f489c069730 con 0x7f489c06b730 2024-04-25T12:06:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.509+0000 7f48927fc700 1 -- 172.21.15.19:0/917802032 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f489809dae0 con 0x7f489c06b730 2024-04-25T12:06:14.508 INFO:teuthology.orchestra.run.smithi019.stdout:287762808870 2024-04-25T12:06:14.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.511+0000 7f48a34c1700 1 -- 172.21.15.19:0/917802032 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f48880a3cd0 msgr2=0x7f48880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.511+0000 7f48a34c1700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f48880a3cd0 0x7f48880a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f488c005950 tx=0x7f488c0058e0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.511+0000 7f48a34c1700 1 -- 172.21.15.19:0/917802032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f489c06b730 msgr2=0x7f489c1a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.511+0000 7f48a34c1700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f489c06b730 0x7f489c1a5410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f48980047b0 tx=0x7f4898009f90 comp rx=0 tx=0).stop 2024-04-25T12:06:14.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.511+0000 7f48a34c1700 1 -- 172.21.15.19:0/917802032 shutdown_connections 2024-04-25T12:06:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.511+0000 7f48a34c1700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f48880a3cd0 0x7f48880a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.511+0000 7f48a34c1700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f489c06b730 0x7f489c1a5410 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.511+0000 7f48a34c1700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f489c1079a0 0x7f489c19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.511+0000 7f48a34c1700 1 --2- 172.21.15.19:0/917802032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f489c1054b0 0x7f489c1a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.511+0000 7f48a34c1700 1 -- 172.21.15.19:0/917802032 >> 172.21.15.19:0/917802032 conn(0x7f489c0789e0 msgr2=0x7f489c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.511+0000 7f48a34c1700 1 -- 172.21.15.19:0/917802032 shutdown_connections 2024-04-25T12:06:14.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:14.511+0000 7f48a34c1700 1 -- 172.21.15.19:0/917802032 wait complete. 2024-04-25T12:06:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.909+0000 7f3343a2f700 1 -- 172.21.15.19:0/3754396895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c102300 msgr2=0x7f333c10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.909+0000 7f3343a2f700 1 --2- 172.21.15.19:0/3754396895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c102300 0x7f333c10e7f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f333400b0a0 tx=0x7f333400b3b0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.909+0000 7f3343a2f700 1 -- 172.21.15.19:0/3754396895 shutdown_connections 2024-04-25T12:06:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.909+0000 7f3343a2f700 1 --2- 172.21.15.19:0/3754396895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c102300 0x7f333c10e7f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.909+0000 7f3343a2f700 1 --2- 172.21.15.19:0/3754396895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f333c107990 0x7f333c107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.909+0000 7f3343a2f700 1 --2- 172.21.15.19:0/3754396895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f333c101940 0x7f333c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.909+0000 7f3343a2f700 1 -- 172.21.15.19:0/3754396895 >> 172.21.15.19:0/3754396895 conn(0x7f333c0788f0 msgr2=0x7f333c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.909+0000 7f3343a2f700 1 -- 172.21.15.19:0/3754396895 shutdown_connections 2024-04-25T12:06:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.909+0000 7f3343a2f700 1 -- 172.21.15.19:0/3754396895 wait complete. 2024-04-25T12:06:14.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3343a2f700 1 Processor -- start 2024-04-25T12:06:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3343a2f700 1 -- start start 2024-04-25T12:06:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3343a2f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f333c101940 0x7f333c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3343a2f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c102300 0x7f333c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3343a2f700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f333c107990 0x7f333c19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3343a2f700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f333c111c60 con 0x7f333c102300 2024-04-25T12:06:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3343a2f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f333c111ae0 con 0x7f333c101940 2024-04-25T12:06:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3343a2f700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f333c111de0 con 0x7f333c107990 2024-04-25T12:06:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3340fca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c102300 0x7f333c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3340fca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c102300 0x7f333c1996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57824/0 (socket says 172.21.15.19:57824) 2024-04-25T12:06:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3340fca700 1 -- 172.21.15.19:0/3113131358 learned_addr learned my addr 172.21.15.19:0/3113131358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:14.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f33417cb700 1 --2- 172.21.15.19:0/3113131358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f333c101940 0x7f333c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:14.640 INFO:teuthology.orchestra.run.smithi019.stdout:197568495668 2024-04-25T12:06:14.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3341fcc700 1 --2- 172.21.15.19:0/3113131358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f333c107990 0x7f333c19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:14.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3340fca700 1 -- 172.21.15.19:0/3113131358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f333c107990 msgr2=0x7f333c19d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3340fca700 1 --2- 172.21.15.19:0/3113131358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f333c107990 0x7f333c19d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3340fca700 1 -- 172.21.15.19:0/3113131358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f333c101940 msgr2=0x7f333c199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3340fca700 1 --2- 172.21.15.19:0/3113131358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f333c101940 0x7f333c199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.910+0000 7f3340fca700 1 -- 172.21.15.19:0/3113131358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c19e120 con 0x7f333c102300 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.911+0000 7f3340fca700 1 --2- 172.21.15.19:0/3113131358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c102300 0x7f333c1996a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f332c00d950 tx=0x7f332c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.911+0000 7f33327fc700 1 -- 172.21.15.19:0/3113131358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c016070 con 0x7f333c102300 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.911+0000 7f33327fc700 1 -- 172.21.15.19:0/3113131358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f332c009e10 con 0x7f333c102300 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.911+0000 7f3343a2f700 1 -- 172.21.15.19:0/3113131358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333c19e410 con 0x7f333c102300 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.911+0000 7f33327fc700 1 -- 172.21.15.19:0/3113131358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c00f620 con 0x7f333c102300 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.911+0000 7f3343a2f700 1 -- 172.21.15.19:0/3113131358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f333c1a5c90 con 0x7f333c102300 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.912+0000 7f3343a2f700 1 -- 172.21.15.19:0/3113131358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333c069730 con 0x7f333c102300 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.913+0000 7f33327fc700 1 -- 172.21.15.19:0/3113131358 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f332c019070 con 0x7f333c102300 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.913+0000 7f33327fc700 1 --2- 172.21.15.19:0/3113131358 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f33280a3d50 0x7f33280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.914+0000 7f33417cb700 1 --2- 172.21.15.19:0/3113131358 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f33280a3d50 0x7f33280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.914+0000 7f33327fc700 1 -- 172.21.15.19:0/3113131358 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f332c0d00c0 con 0x7f333c102300 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.914+0000 7f33417cb700 1 --2- 172.21.15.19:0/3113131358 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f33280a3d50 0x7f33280a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f333800abd0 tx=0x7f333800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:10.917+0000 7f33327fc700 1 -- 172.21.15.19:0/3113131358 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f332c09c920 con 0x7f333c102300 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.131+0000 7f3343a2f700 1 -- 172.21.15.19:0/3113131358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f333c02df80 con 0x7f333c102300 2024-04-25T12:06:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.131+0000 7f33327fc700 1 -- 172.21.15.19:0/3113131358 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f332c09c220 con 0x7f333c102300 2024-04-25T12:06:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.134+0000 7f3343a2f700 1 -- 172.21.15.19:0/3113131358 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f33280a3d50 msgr2=0x7f33280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.134+0000 7f3343a2f700 1 --2- 172.21.15.19:0/3113131358 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f33280a3d50 0x7f33280a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f333800abd0 tx=0x7f333800a380 comp rx=0 tx=0).stop 2024-04-25T12:06:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.134+0000 7f3343a2f700 1 -- 172.21.15.19:0/3113131358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c102300 msgr2=0x7f333c1996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.134+0000 7f3343a2f700 1 --2- 172.21.15.19:0/3113131358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c102300 0x7f333c1996a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f332c00d950 tx=0x7f332c00dc60 comp rx=0 tx=0).stop 2024-04-25T12:06:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.134+0000 7f3343a2f700 1 -- 172.21.15.19:0/3113131358 shutdown_connections 2024-04-25T12:06:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.134+0000 7f3343a2f700 1 --2- 172.21.15.19:0/3113131358 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f33280a3d50 0x7f33280a6210 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.134+0000 7f3343a2f700 1 --2- 172.21.15.19:0/3113131358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f333c107990 0x7f333c19d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.134+0000 7f3343a2f700 1 --2- 172.21.15.19:0/3113131358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f333c101940 0x7f333c199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.134+0000 7f3343a2f700 1 --2- 172.21.15.19:0/3113131358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f333c102300 0x7f333c1996a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.134+0000 7f3343a2f700 1 -- 172.21.15.19:0/3113131358 >> 172.21.15.19:0/3113131358 conn(0x7f333c0788f0 msgr2=0x7f333c06a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.134+0000 7f3343a2f700 1 -- 172.21.15.19:0/3113131358 shutdown_connections 2024-04-25T12:06:14.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:11.134+0000 7f3343a2f700 1 -- 172.21.15.19:0/3113131358 wait complete. 2024-04-25T12:06:14.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:14 smithi019 ceph-mon[29213]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T12:06:14.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:14 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/917802032' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T12:06:14.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:14 smithi169 ceph-mon[32710]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T12:06:14.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:14 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/917802032' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T12:06:14.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:14 smithi150 ceph-mon[32250]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T12:06:14.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:14 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/917802032' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T12:06:15.366 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495664 got 197568495668 for osd.6 2024-04-25T12:06:15.366 DEBUG:teuthology.parallel:result is None 2024-04-25T12:06:16.008 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645338 got 309237645343 for osd.10 2024-04-25T12:06:16.008 DEBUG:teuthology.parallel:result is None 2024-04-25T12:06:16.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:16 smithi019 ceph-mon[29213]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:06:16.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:16 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3821715628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T12:06:16.801 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808865 got 287762808870 for osd.9 2024-04-25T12:06:16.802 DEBUG:teuthology.parallel:result is None 2024-04-25T12:06:16.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:16 smithi169 ceph-mon[32710]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:06:16.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:16 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3821715628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T12:06:16.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:16 smithi150 ceph-mon[32250]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:06:16.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:16 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3821715628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T12:06:17.312 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383701 got 343597383706 for osd.11 2024-04-25T12:06:17.312 DEBUG:teuthology.parallel:result is None 2024-04-25T12:06:17.630 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920827 got 141733920831 for osd.4 2024-04-25T12:06:17.631 DEBUG:teuthology.parallel:result is None 2024-04-25T12:06:17.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.262+0000 7f8171b55700 1 -- 172.21.15.19:0/992634210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f816c107960 msgr2=0x7f816c107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.262+0000 7f8171b55700 1 --2- 172.21.15.19:0/992634210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f816c107960 0x7f816c107d40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8160009990 tx=0x7f8160009ca0 comp rx=0 tx=0).stop 2024-04-25T12:06:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.262+0000 7f8171b55700 1 -- 172.21.15.19:0/992634210 shutdown_connections 2024-04-25T12:06:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.262+0000 7f8171b55700 1 --2- 172.21.15.19:0/992634210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f816c107960 0x7f816c107d40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.262+0000 7f8171b55700 1 --2- 172.21.15.19:0/992634210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f816c102320 0x7f816c10e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.262+0000 7f8171b55700 1 --2- 172.21.15.19:0/992634210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f816c101960 0x7f816c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.262+0000 7f8171b55700 1 -- 172.21.15.19:0/992634210 >> 172.21.15.19:0/992634210 conn(0x7f816c0789e0 msgr2=0x7f816c078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.262+0000 7f8171b55700 1 -- 172.21.15.19:0/992634210 shutdown_connections 2024-04-25T12:06:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.263+0000 7f8171b55700 1 -- 172.21.15.19:0/992634210 wait complete. 2024-04-25T12:06:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.263+0000 7f8171b55700 1 Processor -- start 2024-04-25T12:06:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.263+0000 7f8171b55700 1 -- start start 2024-04-25T12:06:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f8171b55700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f816c101960 0x7f816c19b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f8171b55700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f816c102320 0x7f816c19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:17.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f8171b55700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f816c107960 0x7f816c19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f8171b55700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f816c111cd0 con 0x7f816c107960 2024-04-25T12:06:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f8171b55700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f816c111b50 con 0x7f816c102320 2024-04-25T12:06:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f8171b55700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f816c111e50 con 0x7f816c101960 2024-04-25T12:06:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f816bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f816c107960 0x7f816c19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f816bfff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f816c107960 0x7f816c19fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50480/0 (socket says 172.21.15.19:50480) 2024-04-25T12:06:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f816bfff700 1 -- 172.21.15.19:0/3821715628 learned_addr learned my addr 172.21.15.19:0/3821715628 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f816b7fe700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f816c101960 0x7f816c19b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f816affd700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f816c102320 0x7f816c19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f816bfff700 1 -- 172.21.15.19:0/3821715628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f816c101960 msgr2=0x7f816c19b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f816bfff700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f816c101960 0x7f816c19b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f816bfff700 1 -- 172.21.15.19:0/3821715628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f816c102320 msgr2=0x7f816c19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f816bfff700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f816c102320 0x7f816c19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f816bfff700 1 -- 172.21.15.19:0/3821715628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c1a0540 con 0x7f816c107960 2024-04-25T12:06:17.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f816affd700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f816c102320 0x7f816c19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.264+0000 7f816bfff700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f816c107960 0x7f816c19fe00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8164007ee0 tx=0x7f816400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.265+0000 7f8168ff9700 1 -- 172.21.15.19:0/3821715628 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816400dbd0 con 0x7f816c107960 2024-04-25T12:06:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.265+0000 7f8168ff9700 1 -- 172.21.15.19:0/3821715628 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f816400dd70 con 0x7f816c107960 2024-04-25T12:06:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.265+0000 7f8171b55700 1 -- 172.21.15.19:0/3821715628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c1a0830 con 0x7f816c107960 2024-04-25T12:06:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.265+0000 7f8168ff9700 1 -- 172.21.15.19:0/3821715628 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164005a50 con 0x7f816c107960 2024-04-25T12:06:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.265+0000 7f8171b55700 1 -- 172.21.15.19:0/3821715628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c1a80b0 con 0x7f816c107960 2024-04-25T12:06:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.266+0000 7f8171b55700 1 -- 172.21.15.19:0/3821715628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816c069730 con 0x7f816c107960 2024-04-25T12:06:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.267+0000 7f8168ff9700 1 -- 172.21.15.19:0/3821715628 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f816401a020 con 0x7f816c107960 2024-04-25T12:06:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.268+0000 7f8168ff9700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f81540a3ef0 0x7f81540a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.268+0000 7f816b7fe700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f81540a3ef0 0x7f81540a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.268+0000 7f8168ff9700 1 -- 172.21.15.19:0/3821715628 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f81640d0310 con 0x7f816c107960 2024-04-25T12:06:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.268+0000 7f816b7fe700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f81540a3ef0 0x7f81540a63b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f816000b5c0 tx=0x7f8160005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.272+0000 7f8168ff9700 1 -- 172.21.15.19:0/3821715628 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8164010070 con 0x7f816c107960 2024-04-25T12:06:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.485+0000 7f8171b55700 1 -- 172.21.15.19:0/3821715628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f816c1a8280 con 0x7f816c107960 2024-04-25T12:06:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.485+0000 7f8168ff9700 1 -- 172.21.15.19:0/3821715628 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f816409cb70 con 0x7f816c107960 2024-04-25T12:06:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.487+0000 7f8171b55700 1 -- 172.21.15.19:0/3821715628 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f81540a3ef0 msgr2=0x7f81540a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.487+0000 7f8171b55700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f81540a3ef0 0x7f81540a63b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f816000b5c0 tx=0x7f8160005fd0 comp rx=0 tx=0).stop 2024-04-25T12:06:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.487+0000 7f8171b55700 1 -- 172.21.15.19:0/3821715628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f816c107960 msgr2=0x7f816c19fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.487+0000 7f8171b55700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f816c107960 0x7f816c19fe00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8164007ee0 tx=0x7f816400d3b0 comp rx=0 tx=0).stop 2024-04-25T12:06:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.487+0000 7f8171b55700 1 -- 172.21.15.19:0/3821715628 shutdown_connections 2024-04-25T12:06:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.487+0000 7f8171b55700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f81540a3ef0 0x7f81540a63b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.487+0000 7f8171b55700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f816c107960 0x7f816c19fe00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.487+0000 7f8171b55700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f816c101960 0x7f816c19b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.487+0000 7f8171b55700 1 --2- 172.21.15.19:0/3821715628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f816c102320 0x7f816c19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:17.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.487+0000 7f8171b55700 1 -- 172.21.15.19:0/3821715628 >> 172.21.15.19:0/3821715628 conn(0x7f816c0789e0 msgr2=0x7f816c06b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:17.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.487+0000 7f8171b55700 1 -- 172.21.15.19:0/3821715628 shutdown_connections 2024-04-25T12:06:17.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:16.487+0000 7f8171b55700 1 -- 172.21.15.19:0/3821715628 wait complete. 2024-04-25T12:06:17.995 INFO:teuthology.orchestra.run.smithi019.stdout:253403070507 2024-04-25T12:06:18.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:18 smithi019 ceph-mon[29213]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T12:06:18.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:18 smithi169 ceph-mon[32710]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T12:06:18.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:18 smithi150 ceph-mon[32250]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T12:06:19.577 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070501 got 253403070507 for osd.8 2024-04-25T12:06:19.578 DEBUG:teuthology.parallel:result is None 2024-04-25T12:06:20.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.707+0000 7fe240a21700 1 -- 172.21.15.19:0/1346525948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe23c1022f0 msgr2=0x7fe23c10e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.707+0000 7fe240a21700 1 --2- 172.21.15.19:0/1346525948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe23c1022f0 0x7fe23c10e7e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe23400b0a0 tx=0x7fe23400b3b0 comp rx=0 tx=0).stop 2024-04-25T12:06:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.707+0000 7fe240a21700 1 -- 172.21.15.19:0/1346525948 shutdown_connections 2024-04-25T12:06:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.707+0000 7fe240a21700 1 --2- 172.21.15.19:0/1346525948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe23c1022f0 0x7fe23c10e7e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.707+0000 7fe240a21700 1 --2- 172.21.15.19:0/1346525948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe23c107980 0x7fe23c107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.707+0000 7fe240a21700 1 --2- 172.21.15.19:0/1346525948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe23c101930 0x7fe23c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.707+0000 7fe240a21700 1 -- 172.21.15.19:0/1346525948 >> 172.21.15.19:0/1346525948 conn(0x7fe23c0788b0 msgr2=0x7fe23c078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.707+0000 7fe240a21700 1 -- 172.21.15.19:0/1346525948 shutdown_connections 2024-04-25T12:06:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.707+0000 7fe240a21700 1 -- 172.21.15.19:0/1346525948 wait complete. 2024-04-25T12:06:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.708+0000 7fe240a21700 1 Processor -- start 2024-04-25T12:06:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.708+0000 7fe240a21700 1 -- start start 2024-04-25T12:06:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.708+0000 7fe240a21700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe23c101930 0x7fe23c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.708+0000 7fe240a21700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe23c1022f0 0x7fe23c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.708+0000 7fe240a21700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe23c107980 0x7fe23c19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.708+0000 7fe240a21700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c111c10 con 0x7fe23c1022f0 2024-04-25T12:06:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.708+0000 7fe240a21700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c111a90 con 0x7fe23c101930 2024-04-25T12:06:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.708+0000 7fe240a21700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c111d90 con 0x7fe23c107980 2024-04-25T12:06:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.708+0000 7fe239d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe23c1022f0 0x7fe23c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.708+0000 7fe239d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe23c1022f0 0x7fe23c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50508/0 (socket says 172.21.15.19:50508) 2024-04-25T12:06:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.708+0000 7fe239d9b700 1 -- 172.21.15.19:0/3110485591 learned_addr learned my addr 172.21.15.19:0/3110485591 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.709+0000 7fe23a59c700 1 --2- 172.21.15.19:0/3110485591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe23c101930 0x7fe23c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.709+0000 7fe239d9b700 1 -- 172.21.15.19:0/3110485591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe23c107980 msgr2=0x7fe23c19da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.709+0000 7fe239d9b700 1 --2- 172.21.15.19:0/3110485591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe23c107980 0x7fe23c19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:20.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.709+0000 7fe239d9b700 1 -- 172.21.15.19:0/3110485591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe23c101930 msgr2=0x7fe23c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:20.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.709+0000 7fe239d9b700 1 --2- 172.21.15.19:0/3110485591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe23c101930 0x7fe23c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:20.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.709+0000 7fe239d9b700 1 -- 172.21.15.19:0/3110485591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23c19e1d0 con 0x7fe23c1022f0 2024-04-25T12:06:20.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.709+0000 7fe239d9b700 1 --2- 172.21.15.19:0/3110485591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe23c1022f0 0x7fe23c199700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe23000c920 tx=0x7fe23000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:20.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.709+0000 7fe22b7fe700 1 -- 172.21.15.19:0/3110485591 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230007b00 con 0x7fe23c1022f0 2024-04-25T12:06:20.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.709+0000 7fe240a21700 1 -- 172.21.15.19:0/3110485591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23c19e4c0 con 0x7fe23c1022f0 2024-04-25T12:06:20.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.709+0000 7fe22b7fe700 1 -- 172.21.15.19:0/3110485591 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe230004d00 con 0x7fe23c1022f0 2024-04-25T12:06:20.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.709+0000 7fe22b7fe700 1 -- 172.21.15.19:0/3110485591 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230005620 con 0x7fe23c1022f0 2024-04-25T12:06:20.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.709+0000 7fe240a21700 1 -- 172.21.15.19:0/3110485591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23c1a5d40 con 0x7fe23c1022f0 2024-04-25T12:06:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.710+0000 7fe240a21700 1 -- 172.21.15.19:0/3110485591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe23c069730 con 0x7fe23c1022f0 2024-04-25T12:06:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.711+0000 7fe22b7fe700 1 -- 172.21.15.19:0/3110485591 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe230004750 con 0x7fe23c1022f0 2024-04-25T12:06:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.712+0000 7fe22b7fe700 1 --2- 172.21.15.19:0/3110485591 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe2240a3cd0 0x7fe2240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.712+0000 7fe23a59c700 1 --2- 172.21.15.19:0/3110485591 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe2240a3cd0 0x7fe2240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.712+0000 7fe22b7fe700 1 -- 172.21.15.19:0/3110485591 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fe2300d02f0 con 0x7fe23c1022f0 2024-04-25T12:06:20.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.713+0000 7fe23a59c700 1 --2- 172.21.15.19:0/3110485591 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe2240a3cd0 0x7fe2240a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe22c005950 tx=0x7fe22c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:20.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.716+0000 7fe22b7fe700 1 -- 172.21.15.19:0/3110485591 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe230014030 con 0x7fe23c1022f0 2024-04-25T12:06:20.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:20 smithi169 ceph-mon[32710]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:06:20.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:20 smithi150 ceph-mon[32250]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:06:20.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.945+0000 7fe240a21700 1 -- 172.21.15.19:0/3110485591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe23c02df80 con 0x7fe23c1022f0 2024-04-25T12:06:20.945 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:20 smithi019 ceph-mon[29213]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:06:20.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.946+0000 7fe22b7fe700 1 -- 172.21.15.19:0/3110485591 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe23009cb50 con 0x7fe23c1022f0 2024-04-25T12:06:20.945 INFO:teuthology.orchestra.run.smithi019.stdout:107374182468 2024-04-25T12:06:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.948+0000 7fe240a21700 1 -- 172.21.15.19:0/3110485591 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe2240a3cd0 msgr2=0x7fe2240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.948+0000 7fe240a21700 1 --2- 172.21.15.19:0/3110485591 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe2240a3cd0 0x7fe2240a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe22c005950 tx=0x7fe22c0058e0 comp rx=0 tx=0).stop 2024-04-25T12:06:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.948+0000 7fe240a21700 1 -- 172.21.15.19:0/3110485591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe23c1022f0 msgr2=0x7fe23c199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.948+0000 7fe240a21700 1 --2- 172.21.15.19:0/3110485591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe23c1022f0 0x7fe23c199700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe23000c920 tx=0x7fe23000cce0 comp rx=0 tx=0).stop 2024-04-25T12:06:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.948+0000 7fe240a21700 1 -- 172.21.15.19:0/3110485591 shutdown_connections 2024-04-25T12:06:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.948+0000 7fe240a21700 1 --2- 172.21.15.19:0/3110485591 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe2240a3cd0 0x7fe2240a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.948+0000 7fe240a21700 1 --2- 172.21.15.19:0/3110485591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe23c107980 0x7fe23c19da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:20.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.948+0000 7fe240a21700 1 --2- 172.21.15.19:0/3110485591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe23c101930 0x7fe23c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.948+0000 7fe240a21700 1 --2- 172.21.15.19:0/3110485591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe23c1022f0 0x7fe23c199700 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.948+0000 7fe240a21700 1 -- 172.21.15.19:0/3110485591 >> 172.21.15.19:0/3110485591 conn(0x7fe23c0788b0 msgr2=0x7fe23c06b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.948+0000 7fe240a21700 1 -- 172.21.15.19:0/3110485591 shutdown_connections 2024-04-25T12:06:20.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:20.948+0000 7fe240a21700 1 -- 172.21.15.19:0/3110485591 wait complete. 2024-04-25T12:06:21.624 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182468 for osd.3 2024-04-25T12:06:21.624 DEBUG:teuthology.parallel:result is None 2024-04-25T12:06:21.624 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T12:06:21.624 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T12:06:21.892 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:06:21.905 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:21 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3110485591' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T12:06:21.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:21 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3110485591' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T12:06:22.015 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:21 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3110485591' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T12:06:22.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:22 smithi169 ceph-mon[32710]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T12:06:22.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:22.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:22 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:22.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:22 smithi150 ceph-mon[32250]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T12:06:22.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:22.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:22 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:22.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:22 smithi019 ceph-mon[29213]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T12:06:22.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:22.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:22 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:23.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.543+0000 7ff404bfd700 1 -- 172.21.15.19:0/3037710122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40006a750 msgr2=0x7ff40006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.543+0000 7ff404bfd700 1 --2- 172.21.15.19:0/3037710122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40006a750 0x7ff40006abb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009990 tx=0x7ff3f4009ca0 comp rx=0 tx=0).stop 2024-04-25T12:06:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.543+0000 7ff404bfd700 1 -- 172.21.15.19:0/3037710122 shutdown_connections 2024-04-25T12:06:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.543+0000 7ff404bfd700 1 --2- 172.21.15.19:0/3037710122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff40006b0f0 0x7ff40010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.543+0000 7ff404bfd700 1 --2- 172.21.15.19:0/3037710122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff400108a30 0x7ff400108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.543+0000 7ff404bfd700 1 --2- 172.21.15.19:0/3037710122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff40006a750 0x7ff40006abb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.543+0000 7ff404bfd700 1 -- 172.21.15.19:0/3037710122 >> 172.21.15.19:0/3037710122 conn(0x7ff4000788f0 msgr2=0x7ff400078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.543+0000 7ff404bfd700 1 -- 172.21.15.19:0/3037710122 shutdown_connections 2024-04-25T12:06:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.543+0000 7ff404bfd700 1 -- 172.21.15.19:0/3037710122 wait complete. 2024-04-25T12:06:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.544+0000 7ff404bfd700 1 Processor -- start 2024-04-25T12:06:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.544+0000 7ff404bfd700 1 -- start start 2024-04-25T12:06:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff404bfd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff40006a750 0x7ff400199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff404bfd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff40006b0f0 0x7ff400199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff404bfd700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff400108a30 0x7ff40019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff404bfd700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff400112df0 con 0x7ff400108a30 2024-04-25T12:06:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff404bfd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff400112c70 con 0x7ff40006a750 2024-04-25T12:06:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff404bfd700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff400112f70 con 0x7ff40006b0f0 2024-04-25T12:06:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff3fed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff400108a30 0x7ff40019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff3fed9d700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff400108a30 0x7ff40019dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40910/0 (socket says 172.21.15.19:40910) 2024-04-25T12:06:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff3fed9d700 1 -- 172.21.15.19:0/2581506690 learned_addr learned my addr 172.21.15.19:0/2581506690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff3fdd9b700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff40006b0f0 0x7ff400199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff3fe59c700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff40006a750 0x7ff400199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff3fed9d700 1 -- 172.21.15.19:0/2581506690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff40006b0f0 msgr2=0x7ff400199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff3fed9d700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff40006b0f0 0x7ff400199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff3fed9d700 1 -- 172.21.15.19:0/2581506690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff40006a750 msgr2=0x7ff400199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff3fed9d700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff40006a750 0x7ff400199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff3fed9d700 1 -- 172.21.15.19:0/2581506690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40019e410 con 0x7ff400108a30 2024-04-25T12:06:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff3fe59c700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff40006a750 0x7ff400199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff3fed9d700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff400108a30 0x7ff40019dcd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800b970 tx=0x7ff3f800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.545+0000 7ff3ef7fe700 1 -- 172.21.15.19:0/2581506690 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8019070 con 0x7ff400108a30 2024-04-25T12:06:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.546+0000 7ff3ef7fe700 1 -- 172.21.15.19:0/2581506690 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff3f800ce00 con 0x7ff400108a30 2024-04-25T12:06:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.546+0000 7ff3ef7fe700 1 -- 172.21.15.19:0/2581506690 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8007370 con 0x7ff400108a30 2024-04-25T12:06:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.546+0000 7ff404bfd700 1 -- 172.21.15.19:0/2581506690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40019e700 con 0x7ff400108a30 2024-04-25T12:06:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.546+0000 7ff404bfd700 1 -- 172.21.15.19:0/2581506690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4001a5f80 con 0x7ff400108a30 2024-04-25T12:06:23.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.546+0000 7ff404bfd700 1 -- 172.21.15.19:0/2581506690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40010b030 con 0x7ff400108a30 2024-04-25T12:06:23.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.548+0000 7ff3ef7fe700 1 -- 172.21.15.19:0/2581506690 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff3f801c070 con 0x7ff400108a30 2024-04-25T12:06:23.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.548+0000 7ff3ef7fe700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff3e80a3d50 0x7ff3e80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:23.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.548+0000 7ff3fe59c700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff3e80a3d50 0x7ff3e80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:23.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.548+0000 7ff3ef7fe700 1 -- 172.21.15.19:0/2581506690 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff3f80cfc50 con 0x7ff400108a30 2024-04-25T12:06:23.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.549+0000 7ff3fe59c700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff3e80a3d50 0x7ff3e80a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00099d0 tx=0x7ff3f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:23.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.552+0000 7ff3ef7fe700 1 -- 172.21.15.19:0/2581506690 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff3f809c5c0 con 0x7ff400108a30 2024-04-25T12:06:23.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.760+0000 7ff404bfd700 1 -- 172.21.15.19:0/2581506690 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff40019a7c0 con 0x7ff3e80a3d50 2024-04-25T12:06:23.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.768+0000 7ff3ef7fe700 1 -- 172.21.15.19:0/2581506690 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259068 (secure 0 0 0) 0x7ff40019a7c0 con 0x7ff3e80a3d50 2024-04-25T12:06:23.771 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:23.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.770+0000 7ff404bfd700 1 -- 172.21.15.19:0/2581506690 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff3e80a3d50 msgr2=0x7ff3e80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:23.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.770+0000 7ff404bfd700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff3e80a3d50 0x7ff3e80a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00099d0 tx=0x7ff3f0008040 comp rx=0 tx=0).stop 2024-04-25T12:06:23.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.770+0000 7ff404bfd700 1 -- 172.21.15.19:0/2581506690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff400108a30 msgr2=0x7ff40019dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:23.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.770+0000 7ff404bfd700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff400108a30 0x7ff40019dcd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800b970 tx=0x7ff3f800bc80 comp rx=0 tx=0).stop 2024-04-25T12:06:23.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.771+0000 7ff404bfd700 1 -- 172.21.15.19:0/2581506690 shutdown_connections 2024-04-25T12:06:23.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.771+0000 7ff404bfd700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff3e80a3d50 0x7ff3e80a6210 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:23.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.771+0000 7ff404bfd700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff400108a30 0x7ff40019dcd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:23.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.771+0000 7ff404bfd700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff40006a750 0x7ff400199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:23.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.771+0000 7ff404bfd700 1 --2- 172.21.15.19:0/2581506690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff40006b0f0 0x7ff400199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:23.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.771+0000 7ff404bfd700 1 -- 172.21.15.19:0/2581506690 >> 172.21.15.19:0/2581506690 conn(0x7ff4000788f0 msgr2=0x7ff4000ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:23.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.771+0000 7ff404bfd700 1 -- 172.21.15.19:0/2581506690 shutdown_connections 2024-04-25T12:06:23.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:23.771+0000 7ff404bfd700 1 -- 172.21.15.19:0/2581506690 wait complete. 2024-04-25T12:06:23.774 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T12:06:23.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:06:23.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:06:23.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:23 smithi169 ceph-mon[32710]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:23.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:06:23.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:06:23.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:23 smithi150 ceph-mon[32250]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:24.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:06:24.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:06:24.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:23 smithi019 ceph-mon[29213]: from='mgr.14152 172.21.15.19:0/2410752766' entity='mgr.x' 2024-04-25T12:06:24.680 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-04-25T12:06:22.914053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658461,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":832,"num_read_kb":565,"num_write":419,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1182030,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365092,"kb_used_data":11700,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576732,"statfs":{"total":1151940427776,"available":1151566573568,"internally_reserved":0,"allocated":11980800,"data_stored":7444947,"data_compressed":182163,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002933"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981415+0000","last_change":"2024-04-25T12:05:04.662228+0000","last_active":"2024-04-25T12:05:55.981415+0000","last_peered":"2024-04-25T12:05:55.981415+0000","last_clean":"2024-04-25T12:05:55.981415+0000","last_became_active":"2024-04-25T12:05:04.661855+0000","last_became_peered":"2024-04-25T12:05:04.661855+0000","last_unstale":"2024-04-25T12:05:55.981415+0000","last_undegraded":"2024-04-25T12:05:55.981415+0000","last_fullsized":"2024-04-25T12:05:55.981415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:41.662330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982372+0000","last_change":"2024-04-25T12:05:04.664103+0000","last_active":"2024-04-25T12:05:55.982372+0000","last_peered":"2024-04-25T12:05:55.982372+0000","last_clean":"2024-04-25T12:05:55.982372+0000","last_became_active":"2024-04-25T12:05:04.661946+0000","last_became_peered":"2024-04-25T12:05:04.661946+0000","last_unstale":"2024-04-25T12:05:55.982372+0000","last_undegraded":"2024-04-25T12:05:55.982372+0000","last_fullsized":"2024-04-25T12:05:55.982372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:08.119278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981888+0000","last_change":"2024-04-25T12:05:04.667608+0000","last_active":"2024-04-25T12:05:55.981888+0000","last_peered":"2024-04-25T12:05:55.981888+0000","last_clean":"2024-04-25T12:05:55.981888+0000","last_became_active":"2024-04-25T12:05:04.667227+0000","last_became_peered":"2024-04-25T12:05:04.667227+0000","last_unstale":"2024-04-25T12:05:55.981888+0000","last_undegraded":"2024-04-25T12:05:55.981888+0000","last_fullsized":"2024-04-25T12:05:55.981888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:17.794368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.980362+0000","last_change":"2024-04-25T12:05:04.671431+0000","last_active":"2024-04-25T12:05:55.980362+0000","last_peered":"2024-04-25T12:05:55.980362+0000","last_clean":"2024-04-25T12:05:55.980362+0000","last_became_active":"2024-04-25T12:05:04.670876+0000","last_became_peered":"2024-04-25T12:05:04.670876+0000","last_unstale":"2024-04-25T12:05:55.980362+0000","last_undegraded":"2024-04-25T12:05:55.980362+0000","last_fullsized":"2024-04-25T12:05:55.980362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:03.564092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979318+0000","last_change":"2024-04-25T12:05:04.665061+0000","last_active":"2024-04-25T12:05:55.979318+0000","last_peered":"2024-04-25T12:05:55.979318+0000","last_clean":"2024-04-25T12:05:55.979318+0000","last_became_active":"2024-04-25T12:05:04.664705+0000","last_became_peered":"2024-04-25T12:05:04.664705+0000","last_unstale":"2024-04-25T12:05:55.979318+0000","last_undegraded":"2024-04-25T12:05:55.979318+0000","last_fullsized":"2024-04-25T12:05:55.979318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:16.434402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982174+0000","last_change":"2024-04-25T12:05:04.668285+0000","last_active":"2024-04-25T12:05:55.982174+0000","last_peered":"2024-04-25T12:05:55.982174+0000","last_clean":"2024-04-25T12:05:55.982174+0000","last_became_active":"2024-04-25T12:05:04.667941+0000","last_became_peered":"2024-04-25T12:05:04.667941+0000","last_unstale":"2024-04-25T12:05:55.982174+0000","last_undegraded":"2024-04-25T12:05:55.982174+0000","last_fullsized":"2024-04-25T12:05:55.982174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:02.527280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.989201+0000","last_change":"2024-04-25T12:05:55.989201+0000","last_active":"2024-04-25T12:05:55.989201+0000","last_peered":"2024-04-25T12:05:55.989201+0000","last_clean":"2024-04-25T12:05:55.989201+0000","last_became_active":"2024-04-25T12:05:55.988801+0000","last_became_peered":"2024-04-25T12:05:55.988801+0000","last_unstale":"2024-04-25T12:05:55.989201+0000","last_undegraded":"2024-04-25T12:05:55.989201+0000","last_fullsized":"2024-04-25T12:05:55.989201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.575647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.986887+0000","last_change":"2024-04-25T12:05:04.672998+0000","last_active":"2024-04-25T12:05:55.986887+0000","last_peered":"2024-04-25T12:05:55.986887+0000","last_clean":"2024-04-25T12:05:55.986887+0000","last_became_active":"2024-04-25T12:05:04.672611+0000","last_became_peered":"2024-04-25T12:05:04.672611+0000","last_unstale":"2024-04-25T12:05:55.986887+0000","last_undegraded":"2024-04-25T12:05:55.986887+0000","last_fullsized":"2024-04-25T12:05:55.986887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:22.926657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.988145+0000","last_change":"2024-04-25T12:05:04.662957+0000","last_active":"2024-04-25T12:05:55.988145+0000","last_peered":"2024-04-25T12:05:55.988145+0000","last_clean":"2024-04-25T12:05:55.988145+0000","last_became_active":"2024-04-25T12:05:04.662636+0000","last_became_peered":"2024-04-25T12:05:04.662636+0000","last_unstale":"2024-04-25T12:05:55.988145+0000","last_undegraded":"2024-04-25T12:05:55.988145+0000","last_fullsized":"2024-04-25T12:05:55.988145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:05.209057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987936+0000","last_change":"2024-04-25T12:05:55.987936+0000","last_active":"2024-04-25T12:05:55.987936+0000","last_peered":"2024-04-25T12:05:55.987936+0000","last_clean":"2024-04-25T12:05:55.987936+0000","last_became_active":"2024-04-25T12:05:55.987173+0000","last_became_peered":"2024-04-25T12:05:55.987173+0000","last_unstale":"2024-04-25T12:05:55.987936+0000","last_undegraded":"2024-04-25T12:05:55.987936+0000","last_fullsized":"2024-04-25T12:05:55.987936+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:51.973237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983611+0000","last_change":"2024-04-25T12:05:04.664849+0000","last_active":"2024-04-25T12:05:55.983611+0000","last_peered":"2024-04-25T12:05:55.983611+0000","last_clean":"2024-04-25T12:05:55.983611+0000","last_became_active":"2024-04-25T12:05:04.664322+0000","last_became_peered":"2024-04-25T12:05:04.664322+0000","last_unstale":"2024-04-25T12:05:55.983611+0000","last_undegraded":"2024-04-25T12:05:55.983611+0000","last_fullsized":"2024-04-25T12:05:55.983611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:59:03.872076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.980246+0000","last_change":"2024-04-25T12:05:04.669403+0000","last_active":"2024-04-25T12:05:55.980246+0000","last_peered":"2024-04-25T12:05:55.980246+0000","last_clean":"2024-04-25T12:05:55.980246+0000","last_became_active":"2024-04-25T12:05:04.669052+0000","last_became_peered":"2024-04-25T12:05:04.669052+0000","last_unstale":"2024-04-25T12:05:55.980246+0000","last_undegraded":"2024-04-25T12:05:55.980246+0000","last_fullsized":"2024-04-25T12:05:55.980246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:18.997019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982881+0000","last_change":"2024-04-25T12:05:04.664312+0000","last_active":"2024-04-25T12:05:55.982881+0000","last_peered":"2024-04-25T12:05:55.982881+0000","last_clean":"2024-04-25T12:05:55.982881+0000","last_became_active":"2024-04-25T12:05:04.663980+0000","last_became_peered":"2024-04-25T12:05:04.663980+0000","last_unstale":"2024-04-25T12:05:55.982881+0000","last_undegraded":"2024-04-25T12:05:55.982881+0000","last_fullsized":"2024-04-25T12:05:55.982881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:58.427231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981846+0000","last_change":"2024-04-25T12:05:04.663328+0000","last_active":"2024-04-25T12:05:55.981846+0000","last_peered":"2024-04-25T12:05:55.981846+0000","last_clean":"2024-04-25T12:05:55.981846+0000","last_became_active":"2024-04-25T12:05:04.662575+0000","last_became_peered":"2024-04-25T12:05:04.662575+0000","last_unstale":"2024-04-25T12:05:55.981846+0000","last_undegraded":"2024-04-25T12:05:55.981846+0000","last_fullsized":"2024-04-25T12:05:55.981846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:41.786213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981382+0000","last_change":"2024-04-25T12:05:04.672332+0000","last_active":"2024-04-25T12:05:55.981382+0000","last_peered":"2024-04-25T12:05:55.981382+0000","last_clean":"2024-04-25T12:05:55.981382+0000","last_became_active":"2024-04-25T12:05:04.671951+0000","last_became_peered":"2024-04-25T12:05:04.671951+0000","last_unstale":"2024-04-25T12:05:55.981382+0000","last_undegraded":"2024-04-25T12:05:55.981382+0000","last_fullsized":"2024-04-25T12:05:55.981382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:27.283686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987452+0000","last_change":"2024-04-25T12:05:04.669224+0000","last_active":"2024-04-25T12:05:55.987452+0000","last_peered":"2024-04-25T12:05:55.987452+0000","last_clean":"2024-04-25T12:05:55.987452+0000","last_became_active":"2024-04-25T12:05:04.668894+0000","last_became_peered":"2024-04-25T12:05:04.668894+0000","last_unstale":"2024-04-25T12:05:55.987452+0000","last_undegraded":"2024-04-25T12:05:55.987452+0000","last_fullsized":"2024-04-25T12:05:55.987452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:19.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981508+0000","last_change":"2024-04-25T12:05:04.670391+0000","last_active":"2024-04-25T12:05:55.981508+0000","last_peered":"2024-04-25T12:05:55.981508+0000","last_clean":"2024-04-25T12:05:55.981508+0000","last_became_active":"2024-04-25T12:05:04.670021+0000","last_became_peered":"2024-04-25T12:05:04.670021+0000","last_unstale":"2024-04-25T12:05:55.981508+0000","last_undegraded":"2024-04-25T12:05:55.981508+0000","last_fullsized":"2024-04-25T12:05:55.981508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:40.214491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981431+0000","last_change":"2024-04-25T12:05:04.665102+0000","last_active":"2024-04-25T12:05:55.981431+0000","last_peered":"2024-04-25T12:05:55.981431+0000","last_clean":"2024-04-25T12:05:55.981431+0000","last_became_active":"2024-04-25T12:05:04.664719+0000","last_became_peered":"2024-04-25T12:05:04.664719+0000","last_unstale":"2024-04-25T12:05:55.981431+0000","last_undegraded":"2024-04-25T12:05:55.981431+0000","last_fullsized":"2024-04-25T12:05:55.981431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:37.528031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984700+0000","last_change":"2024-04-25T12:05:04.667931+0000","last_active":"2024-04-25T12:05:55.984700+0000","last_peered":"2024-04-25T12:05:55.984700+0000","last_clean":"2024-04-25T12:05:55.984700+0000","last_became_active":"2024-04-25T12:05:04.667606+0000","last_became_peered":"2024-04-25T12:05:04.667606+0000","last_unstale":"2024-04-25T12:05:55.984700+0000","last_undegraded":"2024-04-25T12:05:55.984700+0000","last_fullsized":"2024-04-25T12:05:55.984700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:43.265302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":123,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:20.021545+0000","last_change":"2024-04-25T12:05:56.006687+0000","last_active":"2024-04-25T12:06:20.021545+0000","last_peered":"2024-04-25T12:06:20.021545+0000","last_clean":"2024-04-25T12:06:20.021545+0000","last_became_active":"2024-04-25T12:05:55.989417+0000","last_became_peered":"2024-04-25T12:05:55.989417+0000","last_unstale":"2024-04-25T12:06:20.021545+0000","last_undegraded":"2024-04-25T12:06:20.021545+0000","last_fullsized":"2024-04-25T12:06:20.021545+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:20.081125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987292+0000","last_change":"2024-04-25T12:05:04.668880+0000","last_active":"2024-04-25T12:05:55.987292+0000","last_peered":"2024-04-25T12:05:55.987292+0000","last_clean":"2024-04-25T12:05:55.987292+0000","last_became_active":"2024-04-25T12:05:04.668463+0000","last_became_peered":"2024-04-25T12:05:04.668463+0000","last_unstale":"2024-04-25T12:05:55.987292+0000","last_undegraded":"2024-04-25T12:05:55.987292+0000","last_fullsized":"2024-04-25T12:05:55.987292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:52.647241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987499+0000","last_change":"2024-04-25T12:05:03.663928+0000","last_active":"2024-04-25T12:05:55.987499+0000","last_peered":"2024-04-25T12:05:55.987499+0000","last_clean":"2024-04-25T12:05:55.987499+0000","last_became_active":"2024-04-25T12:05:03.663611+0000","last_became_peered":"2024-04-25T12:05:03.663611+0000","last_unstale":"2024-04-25T12:05:55.987499+0000","last_undegraded":"2024-04-25T12:05:55.987499+0000","last_fullsized":"2024-04-25T12:05:55.987499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:26.986245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982534+0000","last_change":"2024-04-25T12:05:04.674426+0000","last_active":"2024-04-25T12:05:55.982534+0000","last_peered":"2024-04-25T12:05:55.982534+0000","last_clean":"2024-04-25T12:05:55.982534+0000","last_became_active":"2024-04-25T12:05:04.668668+0000","last_became_peered":"2024-04-25T12:05:04.668668+0000","last_unstale":"2024-04-25T12:05:55.982534+0000","last_undegraded":"2024-04-25T12:05:55.982534+0000","last_fullsized":"2024-04-25T12:05:55.982534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:34.205038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.999811+0000","last_change":"2024-04-25T12:05:55.999327+0000","last_active":"2024-04-25T12:05:55.999811+0000","last_peered":"2024-04-25T12:05:55.999811+0000","last_clean":"2024-04-25T12:05:55.999811+0000","last_became_active":"2024-04-25T12:05:55.986390+0000","last_became_peered":"2024-04-25T12:05:55.986390+0000","last_unstale":"2024-04-25T12:05:55.999811+0000","last_undegraded":"2024-04-25T12:05:55.999811+0000","last_fullsized":"2024-04-25T12:05:55.999811+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.093555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987660+0000","last_change":"2024-04-25T12:05:04.662444+0000","last_active":"2024-04-25T12:05:55.987660+0000","last_peered":"2024-04-25T12:05:55.987660+0000","last_clean":"2024-04-25T12:05:55.987660+0000","last_became_active":"2024-04-25T12:05:04.659285+0000","last_became_peered":"2024-04-25T12:05:04.659285+0000","last_unstale":"2024-04-25T12:05:55.987660+0000","last_undegraded":"2024-04-25T12:05:55.987660+0000","last_fullsized":"2024-04-25T12:05:55.987660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:32.678020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979620+0000","last_change":"2024-04-25T12:05:03.656911+0000","last_active":"2024-04-25T12:05:55.979620+0000","last_peered":"2024-04-25T12:05:55.979620+0000","last_clean":"2024-04-25T12:05:55.979620+0000","last_became_active":"2024-04-25T12:05:03.656553+0000","last_became_peered":"2024-04-25T12:05:03.656553+0000","last_unstale":"2024-04-25T12:05:55.979620+0000","last_undegraded":"2024-04-25T12:05:55.979620+0000","last_fullsized":"2024-04-25T12:05:55.979620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:24.154910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984585+0000","last_change":"2024-04-25T12:05:04.674119+0000","last_active":"2024-04-25T12:05:55.984585+0000","last_peered":"2024-04-25T12:05:55.984585+0000","last_clean":"2024-04-25T12:05:55.984585+0000","last_became_active":"2024-04-25T12:05:04.673260+0000","last_became_peered":"2024-04-25T12:05:04.673260+0000","last_unstale":"2024-04-25T12:05:55.984585+0000","last_undegraded":"2024-04-25T12:05:55.984585+0000","last_fullsized":"2024-04-25T12:05:55.984585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:24.941232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:18.019943+0000","last_change":"2024-04-25T12:05:56.009065+0000","last_active":"2024-04-25T12:06:18.019943+0000","last_peered":"2024-04-25T12:06:18.019943+0000","last_clean":"2024-04-25T12:06:18.019943+0000","last_became_active":"2024-04-25T12:05:55.986850+0000","last_became_peered":"2024-04-25T12:05:55.986850+0000","last_unstale":"2024-04-25T12:06:18.019943+0000","last_undegraded":"2024-04-25T12:06:18.019943+0000","last_fullsized":"2024-04-25T12:06:18.019943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:37.421810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979338+0000","last_change":"2024-04-25T12:05:04.657620+0000","last_active":"2024-04-25T12:05:55.979338+0000","last_peered":"2024-04-25T12:05:55.979338+0000","last_clean":"2024-04-25T12:05:55.979338+0000","last_became_active":"2024-04-25T12:05:04.657264+0000","last_became_peered":"2024-04-25T12:05:04.657264+0000","last_unstale":"2024-04-25T12:05:55.979338+0000","last_undegraded":"2024-04-25T12:05:55.979338+0000","last_fullsized":"2024-04-25T12:05:55.979338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:08.159694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982159+0000","last_change":"2024-04-25T12:05:03.662345+0000","last_active":"2024-04-25T12:05:55.982159+0000","last_peered":"2024-04-25T12:05:55.982159+0000","last_clean":"2024-04-25T12:05:55.982159+0000","last_became_active":"2024-04-25T12:05:03.661843+0000","last_became_peered":"2024-04-25T12:05:03.661843+0000","last_unstale":"2024-04-25T12:05:55.982159+0000","last_undegraded":"2024-04-25T12:05:55.982159+0000","last_fullsized":"2024-04-25T12:05:55.982159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:03.904192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983438+0000","last_change":"2024-04-25T12:05:04.674954+0000","last_active":"2024-04-25T12:05:55.983438+0000","last_peered":"2024-04-25T12:05:55.983438+0000","last_clean":"2024-04-25T12:05:55.983438+0000","last_became_active":"2024-04-25T12:05:04.673652+0000","last_became_peered":"2024-04-25T12:05:04.673652+0000","last_unstale":"2024-04-25T12:05:55.983438+0000","last_undegraded":"2024-04-25T12:05:55.983438+0000","last_fullsized":"2024-04-25T12:05:55.983438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:26.166918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982453+0000","last_change":"2024-04-25T12:05:03.656638+0000","last_active":"2024-04-25T12:05:55.982453+0000","last_peered":"2024-04-25T12:05:55.982453+0000","last_clean":"2024-04-25T12:05:55.982453+0000","last_became_active":"2024-04-25T12:05:03.656276+0000","last_became_peered":"2024-04-25T12:05:03.656276+0000","last_unstale":"2024-04-25T12:05:55.982453+0000","last_undegraded":"2024-04-25T12:05:55.982453+0000","last_fullsized":"2024-04-25T12:05:55.982453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:30.034633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.986775+0000","last_change":"2024-04-25T12:05:04.674146+0000","last_active":"2024-04-25T12:05:55.986775+0000","last_peered":"2024-04-25T12:05:55.986775+0000","last_clean":"2024-04-25T12:05:55.986775+0000","last_became_active":"2024-04-25T12:05:04.673763+0000","last_became_peered":"2024-04-25T12:05:04.673763+0000","last_unstale":"2024-04-25T12:05:55.986775+0000","last_undegraded":"2024-04-25T12:05:55.986775+0000","last_fullsized":"2024-04-25T12:05:55.986775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:09.121546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983355+0000","last_change":"2024-04-25T12:05:03.658527+0000","last_active":"2024-04-25T12:05:55.983355+0000","last_peered":"2024-04-25T12:05:55.983355+0000","last_clean":"2024-04-25T12:05:55.983355+0000","last_became_active":"2024-04-25T12:05:03.658200+0000","last_became_peered":"2024-04-25T12:05:03.658200+0000","last_unstale":"2024-04-25T12:05:55.983355+0000","last_undegraded":"2024-04-25T12:05:55.983355+0000","last_fullsized":"2024-04-25T12:05:55.983355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:17.367641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981194+0000","last_change":"2024-04-25T12:05:04.666431+0000","last_active":"2024-04-25T12:05:55.981194+0000","last_peered":"2024-04-25T12:05:55.981194+0000","last_clean":"2024-04-25T12:05:55.981194+0000","last_became_active":"2024-04-25T12:05:04.666012+0000","last_became_peered":"2024-04-25T12:05:04.666012+0000","last_unstale":"2024-04-25T12:05:55.981194+0000","last_undegraded":"2024-04-25T12:05:55.981194+0000","last_fullsized":"2024-04-25T12:05:55.981194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:26.183735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":166,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:21.023564+0000","last_change":"2024-04-25T12:05:03.659730+0000","last_active":"2024-04-25T12:06:21.023564+0000","last_peered":"2024-04-25T12:06:21.023564+0000","last_clean":"2024-04-25T12:06:21.023564+0000","last_became_active":"2024-04-25T12:05:03.659359+0000","last_became_peered":"2024-04-25T12:05:03.659359+0000","last_unstale":"2024-04-25T12:06:21.023564+0000","last_undegraded":"2024-04-25T12:06:21.023564+0000","last_fullsized":"2024-04-25T12:06:21.023564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:16.644771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":143,"num_read_kb":143,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.988320+0000","last_change":"2024-04-25T12:05:04.666603+0000","last_active":"2024-04-25T12:05:55.988320+0000","last_peered":"2024-04-25T12:05:55.988320+0000","last_clean":"2024-04-25T12:05:55.988320+0000","last_became_active":"2024-04-25T12:05:04.666129+0000","last_became_peered":"2024-04-25T12:05:04.666129+0000","last_unstale":"2024-04-25T12:05:55.988320+0000","last_undegraded":"2024-04-25T12:05:55.988320+0000","last_fullsized":"2024-04-25T12:05:55.988320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:01.024544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:56.009913+0000","last_change":"2024-04-25T12:05:56.009913+0000","last_active":"2024-04-25T12:05:56.009913+0000","last_peered":"2024-04-25T12:05:56.009913+0000","last_clean":"2024-04-25T12:05:56.009913+0000","last_became_active":"2024-04-25T12:05:55.989596+0000","last_became_peered":"2024-04-25T12:05:55.989596+0000","last_unstale":"2024-04-25T12:05:56.009913+0000","last_undegraded":"2024-04-25T12:05:56.009913+0000","last_fullsized":"2024-04-25T12:05:56.009913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:08.399379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982428+0000","last_change":"2024-04-25T12:05:04.669930+0000","last_active":"2024-04-25T12:05:55.982428+0000","last_peered":"2024-04-25T12:05:55.982428+0000","last_clean":"2024-04-25T12:05:55.982428+0000","last_became_active":"2024-04-25T12:05:04.669584+0000","last_became_peered":"2024-04-25T12:05:04.669584+0000","last_unstale":"2024-04-25T12:05:55.982428+0000","last_undegraded":"2024-04-25T12:05:55.982428+0000","last_fullsized":"2024-04-25T12:05:55.982428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:47.250335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984693+0000","last_change":"2024-04-25T12:05:03.661346+0000","last_active":"2024-04-25T12:05:55.984693+0000","last_peered":"2024-04-25T12:05:55.984693+0000","last_clean":"2024-04-25T12:05:55.984693+0000","last_became_active":"2024-04-25T12:05:03.661023+0000","last_became_peered":"2024-04-25T12:05:03.661023+0000","last_unstale":"2024-04-25T12:05:55.984693+0000","last_undegraded":"2024-04-25T12:05:55.984693+0000","last_fullsized":"2024-04-25T12:05:55.984693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:56.659194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.988070+0000","last_change":"2024-04-25T12:05:04.668589+0000","last_active":"2024-04-25T12:05:55.988070+0000","last_peered":"2024-04-25T12:05:55.988070+0000","last_clean":"2024-04-25T12:05:55.988070+0000","last_became_active":"2024-04-25T12:05:04.668124+0000","last_became_peered":"2024-04-25T12:05:04.668124+0000","last_unstale":"2024-04-25T12:05:55.988070+0000","last_undegraded":"2024-04-25T12:05:55.988070+0000","last_fullsized":"2024-04-25T12:05:55.988070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:35.878775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'30","reported_seq":198,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:21.023558+0000","last_change":"2024-04-25T12:05:03.656287+0000","last_active":"2024-04-25T12:06:21.023558+0000","last_peered":"2024-04-25T12:06:21.023558+0000","last_clean":"2024-04-25T12:06:21.023558+0000","last_became_active":"2024-04-25T12:05:03.655928+0000","last_became_peered":"2024-04-25T12:05:03.655928+0000","last_unstale":"2024-04-25T12:06:21.023558+0000","last_undegraded":"2024-04-25T12:06:21.023558+0000","last_fullsized":"2024-04-25T12:06:21.023558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:53.260268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":135,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979488+0000","last_change":"2024-04-25T12:05:04.662866+0000","last_active":"2024-04-25T12:05:55.979488+0000","last_peered":"2024-04-25T12:05:55.979488+0000","last_clean":"2024-04-25T12:05:55.979488+0000","last_became_active":"2024-04-25T12:05:04.661181+0000","last_became_peered":"2024-04-25T12:05:04.661181+0000","last_unstale":"2024-04-25T12:05:55.979488+0000","last_undegraded":"2024-04-25T12:05:55.979488+0000","last_fullsized":"2024-04-25T12:05:55.979488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:46.630686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.985276+0000","last_change":"2024-04-25T12:05:03.663558+0000","last_active":"2024-04-25T12:05:55.985276+0000","last_peered":"2024-04-25T12:05:55.985276+0000","last_clean":"2024-04-25T12:05:55.985276+0000","last_became_active":"2024-04-25T12:05:03.661650+0000","last_became_peered":"2024-04-25T12:05:03.661650+0000","last_unstale":"2024-04-25T12:05:55.985276+0000","last_undegraded":"2024-04-25T12:05:55.985276+0000","last_fullsized":"2024-04-25T12:05:55.985276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:27.599992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.989021+0000","last_change":"2024-04-25T12:05:55.989021+0000","last_active":"2024-04-25T12:05:55.989021+0000","last_peered":"2024-04-25T12:05:55.989021+0000","last_clean":"2024-04-25T12:05:55.989021+0000","last_became_active":"2024-04-25T12:05:55.988246+0000","last_became_peered":"2024-04-25T12:05:55.988246+0000","last_unstale":"2024-04-25T12:05:55.989021+0000","last_undegraded":"2024-04-25T12:05:55.989021+0000","last_fullsized":"2024-04-25T12:05:55.989021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:34.237743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984111+0000","last_change":"2024-04-25T12:05:03.662922+0000","last_active":"2024-04-25T12:05:55.984111+0000","last_peered":"2024-04-25T12:05:55.984111+0000","last_clean":"2024-04-25T12:05:55.984111+0000","last_became_active":"2024-04-25T12:05:03.662584+0000","last_became_peered":"2024-04-25T12:05:03.662584+0000","last_unstale":"2024-04-25T12:05:55.984111+0000","last_undegraded":"2024-04-25T12:05:55.984111+0000","last_fullsized":"2024-04-25T12:05:55.984111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:41.664030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983664+0000","last_change":"2024-04-25T12:05:04.672018+0000","last_active":"2024-04-25T12:05:55.983664+0000","last_peered":"2024-04-25T12:05:55.983664+0000","last_clean":"2024-04-25T12:05:55.983664+0000","last_became_active":"2024-04-25T12:05:04.671670+0000","last_became_peered":"2024-04-25T12:05:04.671670+0000","last_unstale":"2024-04-25T12:05:55.983664+0000","last_undegraded":"2024-04-25T12:05:55.983664+0000","last_fullsized":"2024-04-25T12:05:55.983664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:45.005941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":100,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:21.308073+0000","last_change":"2024-04-25T12:05:03.656826+0000","last_active":"2024-04-25T12:06:21.308073+0000","last_peered":"2024-04-25T12:06:21.308073+0000","last_clean":"2024-04-25T12:06:21.308073+0000","last_became_active":"2024-04-25T12:05:03.656477+0000","last_became_peered":"2024-04-25T12:05:03.656477+0000","last_unstale":"2024-04-25T12:06:21.308073+0000","last_undegraded":"2024-04-25T12:06:21.308073+0000","last_fullsized":"2024-04-25T12:06:21.308073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:38.111569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983325+0000","last_change":"2024-04-25T12:05:04.671771+0000","last_active":"2024-04-25T12:05:55.983325+0000","last_peered":"2024-04-25T12:05:55.983325+0000","last_clean":"2024-04-25T12:05:55.983325+0000","last_became_active":"2024-04-25T12:05:04.671424+0000","last_became_peered":"2024-04-25T12:05:04.671424+0000","last_unstale":"2024-04-25T12:05:55.983325+0000","last_undegraded":"2024-04-25T12:05:55.983325+0000","last_fullsized":"2024-04-25T12:05:55.983325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:51.327337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979202+0000","last_change":"2024-04-25T12:05:03.655884+0000","last_active":"2024-04-25T12:05:55.979202+0000","last_peered":"2024-04-25T12:05:55.979202+0000","last_clean":"2024-04-25T12:05:55.979202+0000","last_became_active":"2024-04-25T12:05:03.655437+0000","last_became_peered":"2024-04-25T12:05:03.655437+0000","last_unstale":"2024-04-25T12:05:55.979202+0000","last_undegraded":"2024-04-25T12:05:55.979202+0000","last_fullsized":"2024-04-25T12:05:55.979202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:24.497861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984027+0000","last_change":"2024-04-25T12:05:04.674785+0000","last_active":"2024-04-25T12:05:55.984027+0000","last_peered":"2024-04-25T12:05:55.984027+0000","last_clean":"2024-04-25T12:05:55.984027+0000","last_became_active":"2024-04-25T12:05:04.674469+0000","last_became_peered":"2024-04-25T12:05:04.674469+0000","last_unstale":"2024-04-25T12:05:55.984027+0000","last_undegraded":"2024-04-25T12:05:55.984027+0000","last_fullsized":"2024-04-25T12:05:55.984027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:28.616188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979699+0000","last_change":"2024-04-25T12:05:03.653954+0000","last_active":"2024-04-25T12:05:55.979699+0000","last_peered":"2024-04-25T12:05:55.979699+0000","last_clean":"2024-04-25T12:05:55.979699+0000","last_became_active":"2024-04-25T12:05:03.653583+0000","last_became_peered":"2024-04-25T12:05:03.653583+0000","last_unstale":"2024-04-25T12:05:55.979699+0000","last_undegraded":"2024-04-25T12:05:55.979699+0000","last_fullsized":"2024-04-25T12:05:55.979699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:58.007741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981248+0000","last_change":"2024-04-25T12:05:04.669395+0000","last_active":"2024-04-25T12:05:55.981248+0000","last_peered":"2024-04-25T12:05:55.981248+0000","last_clean":"2024-04-25T12:05:55.981248+0000","last_became_active":"2024-04-25T12:05:04.669058+0000","last_became_peered":"2024-04-25T12:05:04.669058+0000","last_unstale":"2024-04-25T12:05:55.981248+0000","last_undegraded":"2024-04-25T12:05:55.981248+0000","last_fullsized":"2024-04-25T12:05:55.981248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:03.195889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981946+0000","last_change":"2024-04-25T12:05:03.658899+0000","last_active":"2024-04-25T12:05:55.981946+0000","last_peered":"2024-04-25T12:05:55.981946+0000","last_clean":"2024-04-25T12:05:55.981946+0000","last_became_active":"2024-04-25T12:05:03.658521+0000","last_became_peered":"2024-04-25T12:05:03.658521+0000","last_unstale":"2024-04-25T12:05:55.981946+0000","last_undegraded":"2024-04-25T12:05:55.981946+0000","last_fullsized":"2024-04-25T12:05:55.981946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:21.296484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982466+0000","last_change":"2024-04-25T12:05:04.665958+0000","last_active":"2024-04-25T12:05:55.982466+0000","last_peered":"2024-04-25T12:05:55.982466+0000","last_clean":"2024-04-25T12:05:55.982466+0000","last_became_active":"2024-04-25T12:05:04.665619+0000","last_became_peered":"2024-04-25T12:05:04.665619+0000","last_unstale":"2024-04-25T12:05:55.982466+0000","last_undegraded":"2024-04-25T12:05:55.982466+0000","last_fullsized":"2024-04-25T12:05:55.982466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:46.837053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:18.019299+0000","last_change":"2024-04-25T12:05:03.661021+0000","last_active":"2024-04-25T12:06:18.019299+0000","last_peered":"2024-04-25T12:06:18.019299+0000","last_clean":"2024-04-25T12:06:18.019299+0000","last_became_active":"2024-04-25T12:05:03.660690+0000","last_became_peered":"2024-04-25T12:05:03.660690+0000","last_unstale":"2024-04-25T12:06:18.019299+0000","last_undegraded":"2024-04-25T12:06:18.019299+0000","last_fullsized":"2024-04-25T12:06:18.019299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:00.645287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982172+0000","last_change":"2024-04-25T12:05:04.669496+0000","last_active":"2024-04-25T12:05:55.982172+0000","last_peered":"2024-04-25T12:05:55.982172+0000","last_clean":"2024-04-25T12:05:55.982172+0000","last_became_active":"2024-04-25T12:05:04.669173+0000","last_became_peered":"2024-04-25T12:05:04.669173+0000","last_unstale":"2024-04-25T12:05:55.982172+0000","last_undegraded":"2024-04-25T12:05:55.982172+0000","last_fullsized":"2024-04-25T12:05:55.982172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:30.527192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983609+0000","last_change":"2024-04-25T12:05:04.673622+0000","last_active":"2024-04-25T12:05:55.983609+0000","last_peered":"2024-04-25T12:05:55.983609+0000","last_clean":"2024-04-25T12:05:55.983609+0000","last_became_active":"2024-04-25T12:05:04.673245+0000","last_became_peered":"2024-04-25T12:05:04.673245+0000","last_unstale":"2024-04-25T12:05:55.983609+0000","last_undegraded":"2024-04-25T12:05:55.983609+0000","last_fullsized":"2024-04-25T12:05:55.983609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:59.180387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:21.307091+0000","last_change":"2024-04-25T12:05:03.665419+0000","last_active":"2024-04-25T12:06:21.307091+0000","last_peered":"2024-04-25T12:06:21.307091+0000","last_clean":"2024-04-25T12:06:21.307091+0000","last_became_active":"2024-04-25T12:05:03.664963+0000","last_became_peered":"2024-04-25T12:05:03.664963+0000","last_unstale":"2024-04-25T12:06:21.307091+0000","last_undegraded":"2024-04-25T12:06:21.307091+0000","last_fullsized":"2024-04-25T12:06:21.307091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:04.074104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987464+0000","last_change":"2024-04-25T12:05:04.674257+0000","last_active":"2024-04-25T12:05:55.987464+0000","last_peered":"2024-04-25T12:05:55.987464+0000","last_clean":"2024-04-25T12:05:55.987464+0000","last_became_active":"2024-04-25T12:05:04.669726+0000","last_became_peered":"2024-04-25T12:05:04.669726+0000","last_unstale":"2024-04-25T12:05:55.987464+0000","last_undegraded":"2024-04-25T12:05:55.987464+0000","last_fullsized":"2024-04-25T12:05:55.987464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:03.304088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.980083+0000","last_change":"2024-04-25T12:05:03.652149+0000","last_active":"2024-04-25T12:05:55.980083+0000","last_peered":"2024-04-25T12:05:55.980083+0000","last_clean":"2024-04-25T12:05:55.980083+0000","last_became_active":"2024-04-25T12:05:03.650216+0000","last_became_peered":"2024-04-25T12:05:03.650216+0000","last_unstale":"2024-04-25T12:05:55.980083+0000","last_undegraded":"2024-04-25T12:05:55.980083+0000","last_fullsized":"2024-04-25T12:05:55.980083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:00.255226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983450+0000","last_change":"2024-04-25T12:03:51.170356+0000","last_active":"2024-04-25T12:05:55.983450+0000","last_peered":"2024-04-25T12:05:55.983450+0000","last_clean":"2024-04-25T12:05:55.983450+0000","last_became_active":"2024-04-25T12:03:51.169795+0000","last_became_peered":"2024-04-25T12:03:51.169795+0000","last_unstale":"2024-04-25T12:05:55.983450+0000","last_undegraded":"2024-04-25T12:05:55.983450+0000","last_fullsized":"2024-04-25T12:05:55.983450+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:00:21.810858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:00:21.810858+0000","last_clean_scrub_stamp":"2024-04-25T12:00:21.810858+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:34.246289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981579+0000","last_change":"2024-04-25T12:05:04.671882+0000","last_active":"2024-04-25T12:05:55.981579+0000","last_peered":"2024-04-25T12:05:55.981579+0000","last_clean":"2024-04-25T12:05:55.981579+0000","last_became_active":"2024-04-25T12:05:04.671408+0000","last_became_peered":"2024-04-25T12:05:04.671408+0000","last_unstale":"2024-04-25T12:05:55.981579+0000","last_undegraded":"2024-04-25T12:05:55.981579+0000","last_fullsized":"2024-04-25T12:05:55.981579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:36.465054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983662+0000","last_change":"2024-04-25T12:05:03.659082+0000","last_active":"2024-04-25T12:05:55.983662+0000","last_peered":"2024-04-25T12:05:55.983662+0000","last_clean":"2024-04-25T12:05:55.983662+0000","last_became_active":"2024-04-25T12:05:03.658677+0000","last_became_peered":"2024-04-25T12:05:03.658677+0000","last_unstale":"2024-04-25T12:05:55.983662+0000","last_undegraded":"2024-04-25T12:05:55.983662+0000","last_fullsized":"2024-04-25T12:05:55.983662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:20.938394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979547+0000","last_change":"2024-04-25T12:05:03.661172+0000","last_active":"2024-04-25T12:05:55.979547+0000","last_peered":"2024-04-25T12:05:55.979547+0000","last_clean":"2024-04-25T12:05:55.979547+0000","last_became_active":"2024-04-25T12:05:03.660799+0000","last_became_peered":"2024-04-25T12:05:03.660799+0000","last_unstale":"2024-04-25T12:05:55.979547+0000","last_undegraded":"2024-04-25T12:05:55.979547+0000","last_fullsized":"2024-04-25T12:05:55.979547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:01.889714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982555+0000","last_change":"2024-04-25T12:05:04.662203+0000","last_active":"2024-04-25T12:05:55.982555+0000","last_peered":"2024-04-25T12:05:55.982555+0000","last_clean":"2024-04-25T12:05:55.982555+0000","last_became_active":"2024-04-25T12:05:04.661816+0000","last_became_peered":"2024-04-25T12:05:04.661816+0000","last_unstale":"2024-04-25T12:05:55.982555+0000","last_undegraded":"2024-04-25T12:05:55.982555+0000","last_fullsized":"2024-04-25T12:05:55.982555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:06.656668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981384+0000","last_change":"2024-04-25T12:05:03.664741+0000","last_active":"2024-04-25T12:05:55.981384+0000","last_peered":"2024-04-25T12:05:55.981384+0000","last_clean":"2024-04-25T12:05:55.981384+0000","last_became_active":"2024-04-25T12:05:03.664373+0000","last_became_peered":"2024-04-25T12:05:03.664373+0000","last_unstale":"2024-04-25T12:05:55.981384+0000","last_undegraded":"2024-04-25T12:05:55.981384+0000","last_fullsized":"2024-04-25T12:05:55.981384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:03.540606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987448+0000","last_change":"2024-04-25T12:05:04.677748+0000","last_active":"2024-04-25T12:05:55.987448+0000","last_peered":"2024-04-25T12:05:55.987448+0000","last_clean":"2024-04-25T12:05:55.987448+0000","last_became_active":"2024-04-25T12:05:04.674003+0000","last_became_peered":"2024-04-25T12:05:04.674003+0000","last_unstale":"2024-04-25T12:05:55.987448+0000","last_undegraded":"2024-04-25T12:05:55.987448+0000","last_fullsized":"2024-04-25T12:05:55.987448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:24.031333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.980089+0000","last_change":"2024-04-25T12:05:03.666295+0000","last_active":"2024-04-25T12:05:55.980089+0000","last_peered":"2024-04-25T12:05:55.980089+0000","last_clean":"2024-04-25T12:05:55.980089+0000","last_became_active":"2024-04-25T12:05:03.665910+0000","last_became_peered":"2024-04-25T12:05:03.665910+0000","last_unstale":"2024-04-25T12:05:55.980089+0000","last_undegraded":"2024-04-25T12:05:55.980089+0000","last_fullsized":"2024-04-25T12:05:55.980089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:13.028799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.988948+0000","last_change":"2024-04-25T12:05:55.988948+0000","last_active":"2024-04-25T12:05:55.988948+0000","last_peered":"2024-04-25T12:05:55.988948+0000","last_clean":"2024-04-25T12:05:55.988948+0000","last_became_active":"2024-04-25T12:05:55.988373+0000","last_became_peered":"2024-04-25T12:05:55.988373+0000","last_unstale":"2024-04-25T12:05:55.988948+0000","last_undegraded":"2024-04-25T12:05:55.988948+0000","last_fullsized":"2024-04-25T12:05:55.988948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:29.663313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987535+0000","last_change":"2024-04-25T12:05:03.663799+0000","last_active":"2024-04-25T12:05:55.987535+0000","last_peered":"2024-04-25T12:05:55.987535+0000","last_clean":"2024-04-25T12:05:55.987535+0000","last_became_active":"2024-04-25T12:05:03.663501+0000","last_became_peered":"2024-04-25T12:05:03.663501+0000","last_unstale":"2024-04-25T12:05:55.987535+0000","last_undegraded":"2024-04-25T12:05:55.987535+0000","last_fullsized":"2024-04-25T12:05:55.987535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:34.402036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979202+0000","last_change":"2024-04-25T12:05:04.662594+0000","last_active":"2024-04-25T12:05:55.979202+0000","last_peered":"2024-04-25T12:05:55.979202+0000","last_clean":"2024-04-25T12:05:55.979202+0000","last_became_active":"2024-04-25T12:05:04.662179+0000","last_became_peered":"2024-04-25T12:05:04.662179+0000","last_unstale":"2024-04-25T12:05:55.979202+0000","last_undegraded":"2024-04-25T12:05:55.979202+0000","last_fullsized":"2024-04-25T12:05:55.979202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:58.709837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987161+0000","last_change":"2024-04-25T12:05:03.659416+0000","last_active":"2024-04-25T12:05:55.987161+0000","last_peered":"2024-04-25T12:05:55.987161+0000","last_clean":"2024-04-25T12:05:55.987161+0000","last_became_active":"2024-04-25T12:05:03.658416+0000","last_became_peered":"2024-04-25T12:05:03.658416+0000","last_unstale":"2024-04-25T12:05:55.987161+0000","last_undegraded":"2024-04-25T12:05:55.987161+0000","last_fullsized":"2024-04-25T12:05:55.987161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:01.174438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.980963+0000","last_change":"2024-04-25T12:05:04.670890+0000","last_active":"2024-04-25T12:05:55.980963+0000","last_peered":"2024-04-25T12:05:55.980963+0000","last_clean":"2024-04-25T12:05:55.980963+0000","last_became_active":"2024-04-25T12:05:04.670581+0000","last_became_peered":"2024-04-25T12:05:04.670581+0000","last_unstale":"2024-04-25T12:05:55.980963+0000","last_undegraded":"2024-04-25T12:05:55.980963+0000","last_fullsized":"2024-04-25T12:05:55.980963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.527303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984360+0000","last_change":"2024-04-25T12:05:03.662561+0000","last_active":"2024-04-25T12:05:55.984360+0000","last_peered":"2024-04-25T12:05:55.984360+0000","last_clean":"2024-04-25T12:05:55.984360+0000","last_became_active":"2024-04-25T12:05:03.662155+0000","last_became_peered":"2024-04-25T12:05:03.662155+0000","last_unstale":"2024-04-25T12:05:55.984360+0000","last_undegraded":"2024-04-25T12:05:55.984360+0000","last_fullsized":"2024-04-25T12:05:55.984360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:02.130271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987327+0000","last_change":"2024-04-25T12:05:04.669905+0000","last_active":"2024-04-25T12:05:55.987327+0000","last_peered":"2024-04-25T12:05:55.987327+0000","last_clean":"2024-04-25T12:05:55.987327+0000","last_became_active":"2024-04-25T12:05:04.669322+0000","last_became_peered":"2024-04-25T12:05:04.669322+0000","last_unstale":"2024-04-25T12:05:55.987327+0000","last_undegraded":"2024-04-25T12:05:55.987327+0000","last_fullsized":"2024-04-25T12:05:55.987327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.705643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979364+0000","last_change":"2024-04-25T12:05:03.657923+0000","last_active":"2024-04-25T12:05:55.979364+0000","last_peered":"2024-04-25T12:05:55.979364+0000","last_clean":"2024-04-25T12:05:55.979364+0000","last_became_active":"2024-04-25T12:05:03.657332+0000","last_became_peered":"2024-04-25T12:05:03.657332+0000","last_unstale":"2024-04-25T12:05:55.979364+0000","last_undegraded":"2024-04-25T12:05:55.979364+0000","last_fullsized":"2024-04-25T12:05:55.979364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:54.521822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983652+0000","last_change":"2024-04-25T12:05:04.670193+0000","last_active":"2024-04-25T12:05:55.983652+0000","last_peered":"2024-04-25T12:05:55.983652+0000","last_clean":"2024-04-25T12:05:55.983652+0000","last_became_active":"2024-04-25T12:05:04.669720+0000","last_became_peered":"2024-04-25T12:05:04.669720+0000","last_unstale":"2024-04-25T12:05:55.983652+0000","last_undegraded":"2024-04-25T12:05:55.983652+0000","last_fullsized":"2024-04-25T12:05:55.983652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:26.080914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:19.286776+0000","last_change":"2024-04-25T12:05:03.666064+0000","last_active":"2024-04-25T12:06:19.286776+0000","last_peered":"2024-04-25T12:06:19.286776+0000","last_clean":"2024-04-25T12:06:19.286776+0000","last_became_active":"2024-04-25T12:05:03.665749+0000","last_became_peered":"2024-04-25T12:05:03.665749+0000","last_unstale":"2024-04-25T12:06:19.286776+0000","last_undegraded":"2024-04-25T12:06:19.286776+0000","last_fullsized":"2024-04-25T12:06:19.286776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:21.607549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983657+0000","last_change":"2024-04-25T12:05:04.660953+0000","last_active":"2024-04-25T12:05:55.983657+0000","last_peered":"2024-04-25T12:05:55.983657+0000","last_clean":"2024-04-25T12:05:55.983657+0000","last_became_active":"2024-04-25T12:05:04.660577+0000","last_became_peered":"2024-04-25T12:05:04.660577+0000","last_unstale":"2024-04-25T12:05:55.983657+0000","last_undegraded":"2024-04-25T12:05:55.983657+0000","last_fullsized":"2024-04-25T12:05:55.983657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:35.728839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984456+0000","last_change":"2024-04-25T12:05:03.664454+0000","last_active":"2024-04-25T12:05:55.984456+0000","last_peered":"2024-04-25T12:05:55.984456+0000","last_clean":"2024-04-25T12:05:55.984456+0000","last_became_active":"2024-04-25T12:05:03.664064+0000","last_became_peered":"2024-04-25T12:05:03.664064+0000","last_unstale":"2024-04-25T12:05:55.984456+0000","last_undegraded":"2024-04-25T12:05:55.984456+0000","last_fullsized":"2024-04-25T12:05:55.984456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:28.146713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983622+0000","last_change":"2024-04-25T12:05:04.668385+0000","last_active":"2024-04-25T12:05:55.983622+0000","last_peered":"2024-04-25T12:05:55.983622+0000","last_clean":"2024-04-25T12:05:55.983622+0000","last_became_active":"2024-04-25T12:05:04.667995+0000","last_became_peered":"2024-04-25T12:05:04.667995+0000","last_unstale":"2024-04-25T12:05:55.983622+0000","last_undegraded":"2024-04-25T12:05:55.983622+0000","last_fullsized":"2024-04-25T12:05:55.983622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:39.938151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.994786+0000","last_change":"2024-04-25T12:05:55.994786+0000","last_active":"2024-04-25T12:05:55.994786+0000","last_peered":"2024-04-25T12:05:55.994786+0000","last_clean":"2024-04-25T12:05:55.994786+0000","last_became_active":"2024-04-25T12:05:55.990990+0000","last_became_peered":"2024-04-25T12:05:55.990990+0000","last_unstale":"2024-04-25T12:05:55.994786+0000","last_undegraded":"2024-04-25T12:05:55.994786+0000","last_fullsized":"2024-04-25T12:05:55.994786+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:02.526236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983767+0000","last_change":"2024-04-25T12:05:04.662068+0000","last_active":"2024-04-25T12:05:55.983767+0000","last_peered":"2024-04-25T12:05:55.983767+0000","last_clean":"2024-04-25T12:05:55.983767+0000","last_became_active":"2024-04-25T12:05:04.661694+0000","last_became_peered":"2024-04-25T12:05:04.661694+0000","last_unstale":"2024-04-25T12:05:55.983767+0000","last_undegraded":"2024-04-25T12:05:55.983767+0000","last_fullsized":"2024-04-25T12:05:55.983767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:45.200872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981038+0000","last_change":"2024-04-25T12:05:04.662320+0000","last_active":"2024-04-25T12:05:55.981038+0000","last_peered":"2024-04-25T12:05:55.981038+0000","last_clean":"2024-04-25T12:05:55.981038+0000","last_became_active":"2024-04-25T12:05:04.661992+0000","last_became_peered":"2024-04-25T12:05:04.661992+0000","last_unstale":"2024-04-25T12:05:55.981038+0000","last_undegraded":"2024-04-25T12:05:55.981038+0000","last_fullsized":"2024-04-25T12:05:55.981038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:25.898506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984378+0000","last_change":"2024-04-25T12:05:04.671534+0000","last_active":"2024-04-25T12:05:55.984378+0000","last_peered":"2024-04-25T12:05:55.984378+0000","last_clean":"2024-04-25T12:05:55.984378+0000","last_became_active":"2024-04-25T12:05:04.671169+0000","last_became_peered":"2024-04-25T12:05:04.671169+0000","last_unstale":"2024-04-25T12:05:55.984378+0000","last_undegraded":"2024-04-25T12:05:55.984378+0000","last_fullsized":"2024-04-25T12:05:55.984378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:49.972884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984217+0000","last_change":"2024-04-25T12:05:04.668832+0000","last_active":"2024-04-25T12:05:55.984217+0000","last_peered":"2024-04-25T12:05:55.984217+0000","last_clean":"2024-04-25T12:05:55.984217+0000","last_became_active":"2024-04-25T12:05:04.668459+0000","last_became_peered":"2024-04-25T12:05:04.668459+0000","last_unstale":"2024-04-25T12:05:55.984217+0000","last_undegraded":"2024-04-25T12:05:55.984217+0000","last_fullsized":"2024-04-25T12:05:55.984217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:58.509505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982220+0000","last_change":"2024-04-25T12:05:04.671393+0000","last_active":"2024-04-25T12:05:55.982220+0000","last_peered":"2024-04-25T12:05:55.982220+0000","last_clean":"2024-04-25T12:05:55.982220+0000","last_became_active":"2024-04-25T12:05:04.671041+0000","last_became_peered":"2024-04-25T12:05:04.671041+0000","last_unstale":"2024-04-25T12:05:55.982220+0000","last_undegraded":"2024-04-25T12:05:55.982220+0000","last_fullsized":"2024-04-25T12:05:55.982220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:29.428573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.986680+0000","last_change":"2024-04-25T12:05:55.986680+0000","last_active":"2024-04-25T12:05:55.986680+0000","last_peered":"2024-04-25T12:05:55.986680+0000","last_clean":"2024-04-25T12:05:55.986680+0000","last_became_active":"2024-04-25T12:05:55.986252+0000","last_became_peered":"2024-04-25T12:05:55.986252+0000","last_unstale":"2024-04-25T12:05:55.986680+0000","last_undegraded":"2024-04-25T12:05:55.986680+0000","last_fullsized":"2024-04-25T12:05:55.986680+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:13.570601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981722+0000","last_change":"2024-04-25T12:05:04.664852+0000","last_active":"2024-04-25T12:05:55.981722+0000","last_peered":"2024-04-25T12:05:55.981722+0000","last_clean":"2024-04-25T12:05:55.981722+0000","last_became_active":"2024-04-25T12:05:04.663269+0000","last_became_peered":"2024-04-25T12:05:04.663269+0000","last_unstale":"2024-04-25T12:05:55.981722+0000","last_undegraded":"2024-04-25T12:05:55.981722+0000","last_fullsized":"2024-04-25T12:05:55.981722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:57.862378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.985625+0000","last_change":"2024-04-25T12:05:04.666332+0000","last_active":"2024-04-25T12:05:55.985625+0000","last_peered":"2024-04-25T12:05:55.985625+0000","last_clean":"2024-04-25T12:05:55.985625+0000","last_became_active":"2024-04-25T12:05:04.665777+0000","last_became_peered":"2024-04-25T12:05:04.665777+0000","last_unstale":"2024-04-25T12:05:55.985625+0000","last_undegraded":"2024-04-25T12:05:55.985625+0000","last_fullsized":"2024-04-25T12:05:55.985625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:53.912782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987455+0000","last_change":"2024-04-25T12:05:04.678238+0000","last_active":"2024-04-25T12:05:55.987455+0000","last_peered":"2024-04-25T12:05:55.987455+0000","last_clean":"2024-04-25T12:05:55.987455+0000","last_became_active":"2024-04-25T12:05:04.677660+0000","last_became_peered":"2024-04-25T12:05:04.677660+0000","last_unstale":"2024-04-25T12:05:55.987455+0000","last_undegraded":"2024-04-25T12:05:55.987455+0000","last_fullsized":"2024-04-25T12:05:55.987455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:02.916486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983490+0000","last_change":"2024-04-25T12:05:04.666533+0000","last_active":"2024-04-25T12:05:55.983490+0000","last_peered":"2024-04-25T12:05:55.983490+0000","last_clean":"2024-04-25T12:05:55.983490+0000","last_became_active":"2024-04-25T12:05:04.666184+0000","last_became_peered":"2024-04-25T12:05:04.666184+0000","last_unstale":"2024-04-25T12:05:55.983490+0000","last_undegraded":"2024-04-25T12:05:55.983490+0000","last_fullsized":"2024-04-25T12:05:55.983490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:13.089285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987209+0000","last_change":"2024-04-25T12:05:04.672880+0000","last_active":"2024-04-25T12:05:55.987209+0000","last_peered":"2024-04-25T12:05:55.987209+0000","last_clean":"2024-04-25T12:05:55.987209+0000","last_became_active":"2024-04-25T12:05:04.669640+0000","last_became_peered":"2024-04-25T12:05:04.669640+0000","last_unstale":"2024-04-25T12:05:55.987209+0000","last_undegraded":"2024-04-25T12:05:55.987209+0000","last_fullsized":"2024-04-25T12:05:55.987209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:52.293914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987751+0000","last_change":"2024-04-25T12:05:04.664274+0000","last_active":"2024-04-25T12:05:55.987751+0000","last_peered":"2024-04-25T12:05:55.987751+0000","last_clean":"2024-04-25T12:05:55.987751+0000","last_became_active":"2024-04-25T12:05:04.663715+0000","last_became_peered":"2024-04-25T12:05:04.663715+0000","last_unstale":"2024-04-25T12:05:55.987751+0000","last_undegraded":"2024-04-25T12:05:55.987751+0000","last_fullsized":"2024-04-25T12:05:55.987751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:28.298147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982780+0000","last_change":"2024-04-25T12:05:04.669974+0000","last_active":"2024-04-25T12:05:55.982780+0000","last_peered":"2024-04-25T12:05:55.982780+0000","last_clean":"2024-04-25T12:05:55.982780+0000","last_became_active":"2024-04-25T12:05:04.669531+0000","last_became_peered":"2024-04-25T12:05:04.669531+0000","last_unstale":"2024-04-25T12:05:55.982780+0000","last_undegraded":"2024-04-25T12:05:55.982780+0000","last_fullsized":"2024-04-25T12:05:55.982780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:51.579582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981072+0000","last_change":"2024-04-25T12:05:04.670203+0000","last_active":"2024-04-25T12:05:55.981072+0000","last_peered":"2024-04-25T12:05:55.981072+0000","last_clean":"2024-04-25T12:05:55.981072+0000","last_became_active":"2024-04-25T12:05:04.669865+0000","last_became_peered":"2024-04-25T12:05:04.669865+0000","last_unstale":"2024-04-25T12:05:55.981072+0000","last_undegraded":"2024-04-25T12:05:55.981072+0000","last_fullsized":"2024-04-25T12:05:55.981072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:19.860002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281216,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67859,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":696,"num_read_kb":446,"num_write":236,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1294,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":203577,"data_compressed":5586,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":457611,"data_compressed":5773,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1101824,"data_stored":1109455,"data_compressed":43402,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":456919,"data_compressed":5773,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":456625,"data_compressed":5773,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1049815,"data_compressed":41540,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":456636,"data_compressed":5773,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":519267,"data_compressed":7635,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":457975,"data_compressed":5773,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":518815,"data_compressed":7635,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":456461,"data_compressed":5773,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456888,"data_compressed":5773,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":1048480,"data_compressed":41540,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34127,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34127,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34127,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63470,"data_compressed":1862,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63018,"data_compressed":1862,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63173,"data_compressed":1862,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:06:24.684 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T12:06:24.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:24 smithi169 ceph-mon[32710]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T12:06:24.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:24 smithi169 ceph-mon[32710]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:06:24.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:24 smithi150 ceph-mon[32250]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T12:06:24.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:24 smithi150 ceph-mon[32250]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:06:24.968 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:06:25.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:24 smithi019 ceph-mon[29213]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T12:06:25.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:24 smithi019 ceph-mon[29213]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:06:26.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.662+0000 7ffbed174700 1 -- 172.21.15.19:0/3781952652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe806b1f0 msgr2=0x7ffbe806b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.662+0000 7ffbed174700 1 --2- 172.21.15.19:0/3781952652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe806b1f0 0x7ffbe806b670 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009990 tx=0x7ffbdc009ca0 comp rx=0 tx=0).stop 2024-04-25T12:06:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.662+0000 7ffbed174700 1 -- 172.21.15.19:0/3781952652 shutdown_connections 2024-04-25T12:06:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.662+0000 7ffbed174700 1 --2- 172.21.15.19:0/3781952652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbe806bbb0 0x7ffbe810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.662+0000 7ffbed174700 1 --2- 172.21.15.19:0/3781952652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe8108a60 0x7ffbe8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.662+0000 7ffbed174700 1 --2- 172.21.15.19:0/3781952652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe806b1f0 0x7ffbe806b670 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.662+0000 7ffbed174700 1 -- 172.21.15.19:0/3781952652 >> 172.21.15.19:0/3781952652 conn(0x7ffbe80789e0 msgr2=0x7ffbe8078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.662+0000 7ffbed174700 1 -- 172.21.15.19:0/3781952652 shutdown_connections 2024-04-25T12:06:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.662+0000 7ffbed174700 1 -- 172.21.15.19:0/3781952652 wait complete. 2024-04-25T12:06:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.662+0000 7ffbed174700 1 Processor -- start 2024-04-25T12:06:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbed174700 1 -- start start 2024-04-25T12:06:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbed174700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe806b1f0 0x7ffbe8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbed174700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbe806bbb0 0x7ffbe81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbed174700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8108a60 0x7ffbe819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbed174700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8112db0 con 0x7ffbe8108a60 2024-04-25T12:06:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbed174700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8112c30 con 0x7ffbe806b1f0 2024-04-25T12:06:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbed174700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8112f30 con 0x7ffbe806bbb0 2024-04-25T12:06:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbe759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8108a60 0x7ffbe819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbe759e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8108a60 0x7ffbe819db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40930/0 (socket says 172.21.15.19:40930) 2024-04-25T12:06:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbe759e700 1 -- 172.21.15.19:0/4024393837 learned_addr learned my addr 172.21.15.19:0/4024393837 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbe659c700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbe806bbb0 0x7ffbe81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbe6d9d700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe806b1f0 0x7ffbe8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbe759e700 1 -- 172.21.15.19:0/4024393837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbe806bbb0 msgr2=0x7ffbe81997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbe759e700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbe806bbb0 0x7ffbe81997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbe759e700 1 -- 172.21.15.19:0/4024393837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe806b1f0 msgr2=0x7ffbe8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbe759e700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe806b1f0 0x7ffbe8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbe759e700 1 -- 172.21.15.19:0/4024393837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe819e290 con 0x7ffbe8108a60 2024-04-25T12:06:26.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.663+0000 7ffbe6d9d700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe806b1f0 0x7ffbe8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:26.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.664+0000 7ffbe759e700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8108a60 0x7ffbe819db50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000ef30 tx=0x7ffbe000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:26.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.664+0000 7ffbcffff700 1 -- 172.21.15.19:0/4024393837 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00092e0 con 0x7ffbe8108a60 2024-04-25T12:06:26.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.664+0000 7ffbcffff700 1 -- 172.21.15.19:0/4024393837 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ffbe0004bb0 con 0x7ffbe8108a60 2024-04-25T12:06:26.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.664+0000 7ffbed174700 1 -- 172.21.15.19:0/4024393837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe819e520 con 0x7ffbe8108a60 2024-04-25T12:06:26.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.664+0000 7ffbcffff700 1 -- 172.21.15.19:0/4024393837 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0004280 con 0x7ffbe8108a60 2024-04-25T12:06:26.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.664+0000 7ffbed174700 1 -- 172.21.15.19:0/4024393837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe81a5e00 con 0x7ffbe8108a60 2024-04-25T12:06:26.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.665+0000 7ffbed174700 1 -- 172.21.15.19:0/4024393837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe8069730 con 0x7ffbe8108a60 2024-04-25T12:06:26.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.666+0000 7ffbcffff700 1 -- 172.21.15.19:0/4024393837 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ffbe001b030 con 0x7ffbe8108a60 2024-04-25T12:06:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.666+0000 7ffbcffff700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ffbd00a3c40 0x7ffbd00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.667+0000 7ffbe6d9d700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ffbd00a3c40 0x7ffbd00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.667+0000 7ffbcffff700 1 -- 172.21.15.19:0/4024393837 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ffbe00cffa0 con 0x7ffbe8108a60 2024-04-25T12:06:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.667+0000 7ffbe6d9d700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ffbd00a3c40 0x7ffbd00a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009c90 tx=0x7ffbd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:26.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.671+0000 7ffbcffff700 1 -- 172.21.15.19:0/4024393837 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffbe0022020 con 0x7ffbe8108a60 2024-04-25T12:06:26.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:26 smithi019 ceph-mon[29213]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:06:26.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:26 smithi019 ceph-mon[29213]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T12:06:26.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.873+0000 7ffbed174700 1 -- 172.21.15.19:0/4024393837 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffbe819a5e0 con 0x7ffbd00a3c40 2024-04-25T12:06:26.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.881+0000 7ffbcffff700 1 -- 172.21.15.19:0/4024393837 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259068 (secure 0 0 0) 0x7ffbe819a5e0 con 0x7ffbd00a3c40 2024-04-25T12:06:26.883 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:26.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.884+0000 7ffbed174700 1 -- 172.21.15.19:0/4024393837 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ffbd00a3c40 msgr2=0x7ffbd00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:26.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.884+0000 7ffbed174700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ffbd00a3c40 0x7ffbd00a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009c90 tx=0x7ffbd8009260 comp rx=0 tx=0).stop 2024-04-25T12:06:26.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.884+0000 7ffbed174700 1 -- 172.21.15.19:0/4024393837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8108a60 msgr2=0x7ffbe819db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:26.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.884+0000 7ffbed174700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8108a60 0x7ffbe819db50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000ef30 tx=0x7ffbe000c3a0 comp rx=0 tx=0).stop 2024-04-25T12:06:26.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.884+0000 7ffbed174700 1 -- 172.21.15.19:0/4024393837 shutdown_connections 2024-04-25T12:06:26.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.884+0000 7ffbed174700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ffbd00a3c40 0x7ffbd00a6100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:26.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.884+0000 7ffbed174700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffbe8108a60 0x7ffbe819db50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:26.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.884+0000 7ffbed174700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbe806b1f0 0x7ffbe8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:26.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.884+0000 7ffbed174700 1 --2- 172.21.15.19:0/4024393837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbe806bbb0 0x7ffbe81997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:26.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.884+0000 7ffbed174700 1 -- 172.21.15.19:0/4024393837 >> 172.21.15.19:0/4024393837 conn(0x7ffbe80789e0 msgr2=0x7ffbe80ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:26.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.885+0000 7ffbed174700 1 -- 172.21.15.19:0/4024393837 shutdown_connections 2024-04-25T12:06:26.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:26.885+0000 7ffbed174700 1 -- 172.21.15.19:0/4024393837 wait complete. 2024-04-25T12:06:26.886 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-04-25T12:06:26.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:26 smithi169 ceph-mon[32710]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:06:26.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:26 smithi169 ceph-mon[32710]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T12:06:26.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:26 smithi150 ceph-mon[32250]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:06:26.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:26 smithi150 ceph-mon[32250]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T12:06:27.607 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-04-25T12:06:24.914598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658461,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":837,"num_read_kb":565,"num_write":419,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1182030,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365092,"kb_used_data":11700,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576732,"statfs":{"total":1151940427776,"available":1151566573568,"internally_reserved":0,"allocated":11980800,"data_stored":7444947,"data_compressed":182163,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003054"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981415+0000","last_change":"2024-04-25T12:05:04.662228+0000","last_active":"2024-04-25T12:05:55.981415+0000","last_peered":"2024-04-25T12:05:55.981415+0000","last_clean":"2024-04-25T12:05:55.981415+0000","last_became_active":"2024-04-25T12:05:04.661855+0000","last_became_peered":"2024-04-25T12:05:04.661855+0000","last_unstale":"2024-04-25T12:05:55.981415+0000","last_undegraded":"2024-04-25T12:05:55.981415+0000","last_fullsized":"2024-04-25T12:05:55.981415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:41.662330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982372+0000","last_change":"2024-04-25T12:05:04.664103+0000","last_active":"2024-04-25T12:05:55.982372+0000","last_peered":"2024-04-25T12:05:55.982372+0000","last_clean":"2024-04-25T12:05:55.982372+0000","last_became_active":"2024-04-25T12:05:04.661946+0000","last_became_peered":"2024-04-25T12:05:04.661946+0000","last_unstale":"2024-04-25T12:05:55.982372+0000","last_undegraded":"2024-04-25T12:05:55.982372+0000","last_fullsized":"2024-04-25T12:05:55.982372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:08.119278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981888+0000","last_change":"2024-04-25T12:05:04.667608+0000","last_active":"2024-04-25T12:05:55.981888+0000","last_peered":"2024-04-25T12:05:55.981888+0000","last_clean":"2024-04-25T12:05:55.981888+0000","last_became_active":"2024-04-25T12:05:04.667227+0000","last_became_peered":"2024-04-25T12:05:04.667227+0000","last_unstale":"2024-04-25T12:05:55.981888+0000","last_undegraded":"2024-04-25T12:05:55.981888+0000","last_fullsized":"2024-04-25T12:05:55.981888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:17.794368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.980362+0000","last_change":"2024-04-25T12:05:04.671431+0000","last_active":"2024-04-25T12:05:55.980362+0000","last_peered":"2024-04-25T12:05:55.980362+0000","last_clean":"2024-04-25T12:05:55.980362+0000","last_became_active":"2024-04-25T12:05:04.670876+0000","last_became_peered":"2024-04-25T12:05:04.670876+0000","last_unstale":"2024-04-25T12:05:55.980362+0000","last_undegraded":"2024-04-25T12:05:55.980362+0000","last_fullsized":"2024-04-25T12:05:55.980362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:03.564092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979318+0000","last_change":"2024-04-25T12:05:04.665061+0000","last_active":"2024-04-25T12:05:55.979318+0000","last_peered":"2024-04-25T12:05:55.979318+0000","last_clean":"2024-04-25T12:05:55.979318+0000","last_became_active":"2024-04-25T12:05:04.664705+0000","last_became_peered":"2024-04-25T12:05:04.664705+0000","last_unstale":"2024-04-25T12:05:55.979318+0000","last_undegraded":"2024-04-25T12:05:55.979318+0000","last_fullsized":"2024-04-25T12:05:55.979318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:16.434402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982174+0000","last_change":"2024-04-25T12:05:04.668285+0000","last_active":"2024-04-25T12:05:55.982174+0000","last_peered":"2024-04-25T12:05:55.982174+0000","last_clean":"2024-04-25T12:05:55.982174+0000","last_became_active":"2024-04-25T12:05:04.667941+0000","last_became_peered":"2024-04-25T12:05:04.667941+0000","last_unstale":"2024-04-25T12:05:55.982174+0000","last_undegraded":"2024-04-25T12:05:55.982174+0000","last_fullsized":"2024-04-25T12:05:55.982174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:02.527280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.989201+0000","last_change":"2024-04-25T12:05:55.989201+0000","last_active":"2024-04-25T12:05:55.989201+0000","last_peered":"2024-04-25T12:05:55.989201+0000","last_clean":"2024-04-25T12:05:55.989201+0000","last_became_active":"2024-04-25T12:05:55.988801+0000","last_became_peered":"2024-04-25T12:05:55.988801+0000","last_unstale":"2024-04-25T12:05:55.989201+0000","last_undegraded":"2024-04-25T12:05:55.989201+0000","last_fullsized":"2024-04-25T12:05:55.989201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.575647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.986887+0000","last_change":"2024-04-25T12:05:04.672998+0000","last_active":"2024-04-25T12:05:55.986887+0000","last_peered":"2024-04-25T12:05:55.986887+0000","last_clean":"2024-04-25T12:05:55.986887+0000","last_became_active":"2024-04-25T12:05:04.672611+0000","last_became_peered":"2024-04-25T12:05:04.672611+0000","last_unstale":"2024-04-25T12:05:55.986887+0000","last_undegraded":"2024-04-25T12:05:55.986887+0000","last_fullsized":"2024-04-25T12:05:55.986887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:22.926657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.988145+0000","last_change":"2024-04-25T12:05:04.662957+0000","last_active":"2024-04-25T12:05:55.988145+0000","last_peered":"2024-04-25T12:05:55.988145+0000","last_clean":"2024-04-25T12:05:55.988145+0000","last_became_active":"2024-04-25T12:05:04.662636+0000","last_became_peered":"2024-04-25T12:05:04.662636+0000","last_unstale":"2024-04-25T12:05:55.988145+0000","last_undegraded":"2024-04-25T12:05:55.988145+0000","last_fullsized":"2024-04-25T12:05:55.988145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:05.209057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987936+0000","last_change":"2024-04-25T12:05:55.987936+0000","last_active":"2024-04-25T12:05:55.987936+0000","last_peered":"2024-04-25T12:05:55.987936+0000","last_clean":"2024-04-25T12:05:55.987936+0000","last_became_active":"2024-04-25T12:05:55.987173+0000","last_became_peered":"2024-04-25T12:05:55.987173+0000","last_unstale":"2024-04-25T12:05:55.987936+0000","last_undegraded":"2024-04-25T12:05:55.987936+0000","last_fullsized":"2024-04-25T12:05:55.987936+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:51.973237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983611+0000","last_change":"2024-04-25T12:05:04.664849+0000","last_active":"2024-04-25T12:05:55.983611+0000","last_peered":"2024-04-25T12:05:55.983611+0000","last_clean":"2024-04-25T12:05:55.983611+0000","last_became_active":"2024-04-25T12:05:04.664322+0000","last_became_peered":"2024-04-25T12:05:04.664322+0000","last_unstale":"2024-04-25T12:05:55.983611+0000","last_undegraded":"2024-04-25T12:05:55.983611+0000","last_fullsized":"2024-04-25T12:05:55.983611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:59:03.872076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.980246+0000","last_change":"2024-04-25T12:05:04.669403+0000","last_active":"2024-04-25T12:05:55.980246+0000","last_peered":"2024-04-25T12:05:55.980246+0000","last_clean":"2024-04-25T12:05:55.980246+0000","last_became_active":"2024-04-25T12:05:04.669052+0000","last_became_peered":"2024-04-25T12:05:04.669052+0000","last_unstale":"2024-04-25T12:05:55.980246+0000","last_undegraded":"2024-04-25T12:05:55.980246+0000","last_fullsized":"2024-04-25T12:05:55.980246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:18.997019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982881+0000","last_change":"2024-04-25T12:05:04.664312+0000","last_active":"2024-04-25T12:05:55.982881+0000","last_peered":"2024-04-25T12:05:55.982881+0000","last_clean":"2024-04-25T12:05:55.982881+0000","last_became_active":"2024-04-25T12:05:04.663980+0000","last_became_peered":"2024-04-25T12:05:04.663980+0000","last_unstale":"2024-04-25T12:05:55.982881+0000","last_undegraded":"2024-04-25T12:05:55.982881+0000","last_fullsized":"2024-04-25T12:05:55.982881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:58.427231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981846+0000","last_change":"2024-04-25T12:05:04.663328+0000","last_active":"2024-04-25T12:05:55.981846+0000","last_peered":"2024-04-25T12:05:55.981846+0000","last_clean":"2024-04-25T12:05:55.981846+0000","last_became_active":"2024-04-25T12:05:04.662575+0000","last_became_peered":"2024-04-25T12:05:04.662575+0000","last_unstale":"2024-04-25T12:05:55.981846+0000","last_undegraded":"2024-04-25T12:05:55.981846+0000","last_fullsized":"2024-04-25T12:05:55.981846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:41.786213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981382+0000","last_change":"2024-04-25T12:05:04.672332+0000","last_active":"2024-04-25T12:05:55.981382+0000","last_peered":"2024-04-25T12:05:55.981382+0000","last_clean":"2024-04-25T12:05:55.981382+0000","last_became_active":"2024-04-25T12:05:04.671951+0000","last_became_peered":"2024-04-25T12:05:04.671951+0000","last_unstale":"2024-04-25T12:05:55.981382+0000","last_undegraded":"2024-04-25T12:05:55.981382+0000","last_fullsized":"2024-04-25T12:05:55.981382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:27.283686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987452+0000","last_change":"2024-04-25T12:05:04.669224+0000","last_active":"2024-04-25T12:05:55.987452+0000","last_peered":"2024-04-25T12:05:55.987452+0000","last_clean":"2024-04-25T12:05:55.987452+0000","last_became_active":"2024-04-25T12:05:04.668894+0000","last_became_peered":"2024-04-25T12:05:04.668894+0000","last_unstale":"2024-04-25T12:05:55.987452+0000","last_undegraded":"2024-04-25T12:05:55.987452+0000","last_fullsized":"2024-04-25T12:05:55.987452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:19.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981508+0000","last_change":"2024-04-25T12:05:04.670391+0000","last_active":"2024-04-25T12:05:55.981508+0000","last_peered":"2024-04-25T12:05:55.981508+0000","last_clean":"2024-04-25T12:05:55.981508+0000","last_became_active":"2024-04-25T12:05:04.670021+0000","last_became_peered":"2024-04-25T12:05:04.670021+0000","last_unstale":"2024-04-25T12:05:55.981508+0000","last_undegraded":"2024-04-25T12:05:55.981508+0000","last_fullsized":"2024-04-25T12:05:55.981508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:40.214491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981431+0000","last_change":"2024-04-25T12:05:04.665102+0000","last_active":"2024-04-25T12:05:55.981431+0000","last_peered":"2024-04-25T12:05:55.981431+0000","last_clean":"2024-04-25T12:05:55.981431+0000","last_became_active":"2024-04-25T12:05:04.664719+0000","last_became_peered":"2024-04-25T12:05:04.664719+0000","last_unstale":"2024-04-25T12:05:55.981431+0000","last_undegraded":"2024-04-25T12:05:55.981431+0000","last_fullsized":"2024-04-25T12:05:55.981431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:37.528031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984700+0000","last_change":"2024-04-25T12:05:04.667931+0000","last_active":"2024-04-25T12:05:55.984700+0000","last_peered":"2024-04-25T12:05:55.984700+0000","last_clean":"2024-04-25T12:05:55.984700+0000","last_became_active":"2024-04-25T12:05:04.667606+0000","last_became_peered":"2024-04-25T12:05:04.667606+0000","last_unstale":"2024-04-25T12:05:55.984700+0000","last_undegraded":"2024-04-25T12:05:55.984700+0000","last_fullsized":"2024-04-25T12:05:55.984700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:43.265302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":123,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:20.021545+0000","last_change":"2024-04-25T12:05:56.006687+0000","last_active":"2024-04-25T12:06:20.021545+0000","last_peered":"2024-04-25T12:06:20.021545+0000","last_clean":"2024-04-25T12:06:20.021545+0000","last_became_active":"2024-04-25T12:05:55.989417+0000","last_became_peered":"2024-04-25T12:05:55.989417+0000","last_unstale":"2024-04-25T12:06:20.021545+0000","last_undegraded":"2024-04-25T12:06:20.021545+0000","last_fullsized":"2024-04-25T12:06:20.021545+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:20.081125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987292+0000","last_change":"2024-04-25T12:05:04.668880+0000","last_active":"2024-04-25T12:05:55.987292+0000","last_peered":"2024-04-25T12:05:55.987292+0000","last_clean":"2024-04-25T12:05:55.987292+0000","last_became_active":"2024-04-25T12:05:04.668463+0000","last_became_peered":"2024-04-25T12:05:04.668463+0000","last_unstale":"2024-04-25T12:05:55.987292+0000","last_undegraded":"2024-04-25T12:05:55.987292+0000","last_fullsized":"2024-04-25T12:05:55.987292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:52.647241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987499+0000","last_change":"2024-04-25T12:05:03.663928+0000","last_active":"2024-04-25T12:05:55.987499+0000","last_peered":"2024-04-25T12:05:55.987499+0000","last_clean":"2024-04-25T12:05:55.987499+0000","last_became_active":"2024-04-25T12:05:03.663611+0000","last_became_peered":"2024-04-25T12:05:03.663611+0000","last_unstale":"2024-04-25T12:05:55.987499+0000","last_undegraded":"2024-04-25T12:05:55.987499+0000","last_fullsized":"2024-04-25T12:05:55.987499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:26.986245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982534+0000","last_change":"2024-04-25T12:05:04.674426+0000","last_active":"2024-04-25T12:05:55.982534+0000","last_peered":"2024-04-25T12:05:55.982534+0000","last_clean":"2024-04-25T12:05:55.982534+0000","last_became_active":"2024-04-25T12:05:04.668668+0000","last_became_peered":"2024-04-25T12:05:04.668668+0000","last_unstale":"2024-04-25T12:05:55.982534+0000","last_undegraded":"2024-04-25T12:05:55.982534+0000","last_fullsized":"2024-04-25T12:05:55.982534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:34.205038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.999811+0000","last_change":"2024-04-25T12:05:55.999327+0000","last_active":"2024-04-25T12:05:55.999811+0000","last_peered":"2024-04-25T12:05:55.999811+0000","last_clean":"2024-04-25T12:05:55.999811+0000","last_became_active":"2024-04-25T12:05:55.986390+0000","last_became_peered":"2024-04-25T12:05:55.986390+0000","last_unstale":"2024-04-25T12:05:55.999811+0000","last_undegraded":"2024-04-25T12:05:55.999811+0000","last_fullsized":"2024-04-25T12:05:55.999811+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.093555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987660+0000","last_change":"2024-04-25T12:05:04.662444+0000","last_active":"2024-04-25T12:05:55.987660+0000","last_peered":"2024-04-25T12:05:55.987660+0000","last_clean":"2024-04-25T12:05:55.987660+0000","last_became_active":"2024-04-25T12:05:04.659285+0000","last_became_peered":"2024-04-25T12:05:04.659285+0000","last_unstale":"2024-04-25T12:05:55.987660+0000","last_undegraded":"2024-04-25T12:05:55.987660+0000","last_fullsized":"2024-04-25T12:05:55.987660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:32.678020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979620+0000","last_change":"2024-04-25T12:05:03.656911+0000","last_active":"2024-04-25T12:05:55.979620+0000","last_peered":"2024-04-25T12:05:55.979620+0000","last_clean":"2024-04-25T12:05:55.979620+0000","last_became_active":"2024-04-25T12:05:03.656553+0000","last_became_peered":"2024-04-25T12:05:03.656553+0000","last_unstale":"2024-04-25T12:05:55.979620+0000","last_undegraded":"2024-04-25T12:05:55.979620+0000","last_fullsized":"2024-04-25T12:05:55.979620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:24.154910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984585+0000","last_change":"2024-04-25T12:05:04.674119+0000","last_active":"2024-04-25T12:05:55.984585+0000","last_peered":"2024-04-25T12:05:55.984585+0000","last_clean":"2024-04-25T12:05:55.984585+0000","last_became_active":"2024-04-25T12:05:04.673260+0000","last_became_peered":"2024-04-25T12:05:04.673260+0000","last_unstale":"2024-04-25T12:05:55.984585+0000","last_undegraded":"2024-04-25T12:05:55.984585+0000","last_fullsized":"2024-04-25T12:05:55.984585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:24.941232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:18.019943+0000","last_change":"2024-04-25T12:05:56.009065+0000","last_active":"2024-04-25T12:06:18.019943+0000","last_peered":"2024-04-25T12:06:18.019943+0000","last_clean":"2024-04-25T12:06:18.019943+0000","last_became_active":"2024-04-25T12:05:55.986850+0000","last_became_peered":"2024-04-25T12:05:55.986850+0000","last_unstale":"2024-04-25T12:06:18.019943+0000","last_undegraded":"2024-04-25T12:06:18.019943+0000","last_fullsized":"2024-04-25T12:06:18.019943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:37.421810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979338+0000","last_change":"2024-04-25T12:05:04.657620+0000","last_active":"2024-04-25T12:05:55.979338+0000","last_peered":"2024-04-25T12:05:55.979338+0000","last_clean":"2024-04-25T12:05:55.979338+0000","last_became_active":"2024-04-25T12:05:04.657264+0000","last_became_peered":"2024-04-25T12:05:04.657264+0000","last_unstale":"2024-04-25T12:05:55.979338+0000","last_undegraded":"2024-04-25T12:05:55.979338+0000","last_fullsized":"2024-04-25T12:05:55.979338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:08.159694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982159+0000","last_change":"2024-04-25T12:05:03.662345+0000","last_active":"2024-04-25T12:05:55.982159+0000","last_peered":"2024-04-25T12:05:55.982159+0000","last_clean":"2024-04-25T12:05:55.982159+0000","last_became_active":"2024-04-25T12:05:03.661843+0000","last_became_peered":"2024-04-25T12:05:03.661843+0000","last_unstale":"2024-04-25T12:05:55.982159+0000","last_undegraded":"2024-04-25T12:05:55.982159+0000","last_fullsized":"2024-04-25T12:05:55.982159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:03.904192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983438+0000","last_change":"2024-04-25T12:05:04.674954+0000","last_active":"2024-04-25T12:05:55.983438+0000","last_peered":"2024-04-25T12:05:55.983438+0000","last_clean":"2024-04-25T12:05:55.983438+0000","last_became_active":"2024-04-25T12:05:04.673652+0000","last_became_peered":"2024-04-25T12:05:04.673652+0000","last_unstale":"2024-04-25T12:05:55.983438+0000","last_undegraded":"2024-04-25T12:05:55.983438+0000","last_fullsized":"2024-04-25T12:05:55.983438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:26.166918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982453+0000","last_change":"2024-04-25T12:05:03.656638+0000","last_active":"2024-04-25T12:05:55.982453+0000","last_peered":"2024-04-25T12:05:55.982453+0000","last_clean":"2024-04-25T12:05:55.982453+0000","last_became_active":"2024-04-25T12:05:03.656276+0000","last_became_peered":"2024-04-25T12:05:03.656276+0000","last_unstale":"2024-04-25T12:05:55.982453+0000","last_undegraded":"2024-04-25T12:05:55.982453+0000","last_fullsized":"2024-04-25T12:05:55.982453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:30.034633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.986775+0000","last_change":"2024-04-25T12:05:04.674146+0000","last_active":"2024-04-25T12:05:55.986775+0000","last_peered":"2024-04-25T12:05:55.986775+0000","last_clean":"2024-04-25T12:05:55.986775+0000","last_became_active":"2024-04-25T12:05:04.673763+0000","last_became_peered":"2024-04-25T12:05:04.673763+0000","last_unstale":"2024-04-25T12:05:55.986775+0000","last_undegraded":"2024-04-25T12:05:55.986775+0000","last_fullsized":"2024-04-25T12:05:55.986775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:09.121546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983355+0000","last_change":"2024-04-25T12:05:03.658527+0000","last_active":"2024-04-25T12:05:55.983355+0000","last_peered":"2024-04-25T12:05:55.983355+0000","last_clean":"2024-04-25T12:05:55.983355+0000","last_became_active":"2024-04-25T12:05:03.658200+0000","last_became_peered":"2024-04-25T12:05:03.658200+0000","last_unstale":"2024-04-25T12:05:55.983355+0000","last_undegraded":"2024-04-25T12:05:55.983355+0000","last_fullsized":"2024-04-25T12:05:55.983355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:17.367641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981194+0000","last_change":"2024-04-25T12:05:04.666431+0000","last_active":"2024-04-25T12:05:55.981194+0000","last_peered":"2024-04-25T12:05:55.981194+0000","last_clean":"2024-04-25T12:05:55.981194+0000","last_became_active":"2024-04-25T12:05:04.666012+0000","last_became_peered":"2024-04-25T12:05:04.666012+0000","last_unstale":"2024-04-25T12:05:55.981194+0000","last_undegraded":"2024-04-25T12:05:55.981194+0000","last_fullsized":"2024-04-25T12:05:55.981194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:26.183735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":166,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:21.023564+0000","last_change":"2024-04-25T12:05:03.659730+0000","last_active":"2024-04-25T12:06:21.023564+0000","last_peered":"2024-04-25T12:06:21.023564+0000","last_clean":"2024-04-25T12:06:21.023564+0000","last_became_active":"2024-04-25T12:05:03.659359+0000","last_became_peered":"2024-04-25T12:05:03.659359+0000","last_unstale":"2024-04-25T12:06:21.023564+0000","last_undegraded":"2024-04-25T12:06:21.023564+0000","last_fullsized":"2024-04-25T12:06:21.023564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:16.644771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":143,"num_read_kb":143,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.988320+0000","last_change":"2024-04-25T12:05:04.666603+0000","last_active":"2024-04-25T12:05:55.988320+0000","last_peered":"2024-04-25T12:05:55.988320+0000","last_clean":"2024-04-25T12:05:55.988320+0000","last_became_active":"2024-04-25T12:05:04.666129+0000","last_became_peered":"2024-04-25T12:05:04.666129+0000","last_unstale":"2024-04-25T12:05:55.988320+0000","last_undegraded":"2024-04-25T12:05:55.988320+0000","last_fullsized":"2024-04-25T12:05:55.988320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:01.024544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:56.009913+0000","last_change":"2024-04-25T12:05:56.009913+0000","last_active":"2024-04-25T12:05:56.009913+0000","last_peered":"2024-04-25T12:05:56.009913+0000","last_clean":"2024-04-25T12:05:56.009913+0000","last_became_active":"2024-04-25T12:05:55.989596+0000","last_became_peered":"2024-04-25T12:05:55.989596+0000","last_unstale":"2024-04-25T12:05:56.009913+0000","last_undegraded":"2024-04-25T12:05:56.009913+0000","last_fullsized":"2024-04-25T12:05:56.009913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:08.399379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982428+0000","last_change":"2024-04-25T12:05:04.669930+0000","last_active":"2024-04-25T12:05:55.982428+0000","last_peered":"2024-04-25T12:05:55.982428+0000","last_clean":"2024-04-25T12:05:55.982428+0000","last_became_active":"2024-04-25T12:05:04.669584+0000","last_became_peered":"2024-04-25T12:05:04.669584+0000","last_unstale":"2024-04-25T12:05:55.982428+0000","last_undegraded":"2024-04-25T12:05:55.982428+0000","last_fullsized":"2024-04-25T12:05:55.982428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:47.250335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984693+0000","last_change":"2024-04-25T12:05:03.661346+0000","last_active":"2024-04-25T12:05:55.984693+0000","last_peered":"2024-04-25T12:05:55.984693+0000","last_clean":"2024-04-25T12:05:55.984693+0000","last_became_active":"2024-04-25T12:05:03.661023+0000","last_became_peered":"2024-04-25T12:05:03.661023+0000","last_unstale":"2024-04-25T12:05:55.984693+0000","last_undegraded":"2024-04-25T12:05:55.984693+0000","last_fullsized":"2024-04-25T12:05:55.984693+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:56.659194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.988070+0000","last_change":"2024-04-25T12:05:04.668589+0000","last_active":"2024-04-25T12:05:55.988070+0000","last_peered":"2024-04-25T12:05:55.988070+0000","last_clean":"2024-04-25T12:05:55.988070+0000","last_became_active":"2024-04-25T12:05:04.668124+0000","last_became_peered":"2024-04-25T12:05:04.668124+0000","last_unstale":"2024-04-25T12:05:55.988070+0000","last_undegraded":"2024-04-25T12:05:55.988070+0000","last_fullsized":"2024-04-25T12:05:55.988070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:35.878775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'30","reported_seq":198,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:21.023558+0000","last_change":"2024-04-25T12:05:03.656287+0000","last_active":"2024-04-25T12:06:21.023558+0000","last_peered":"2024-04-25T12:06:21.023558+0000","last_clean":"2024-04-25T12:06:21.023558+0000","last_became_active":"2024-04-25T12:05:03.655928+0000","last_became_peered":"2024-04-25T12:05:03.655928+0000","last_unstale":"2024-04-25T12:06:21.023558+0000","last_undegraded":"2024-04-25T12:06:21.023558+0000","last_fullsized":"2024-04-25T12:06:21.023558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:53.260268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":135,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979488+0000","last_change":"2024-04-25T12:05:04.662866+0000","last_active":"2024-04-25T12:05:55.979488+0000","last_peered":"2024-04-25T12:05:55.979488+0000","last_clean":"2024-04-25T12:05:55.979488+0000","last_became_active":"2024-04-25T12:05:04.661181+0000","last_became_peered":"2024-04-25T12:05:04.661181+0000","last_unstale":"2024-04-25T12:05:55.979488+0000","last_undegraded":"2024-04-25T12:05:55.979488+0000","last_fullsized":"2024-04-25T12:05:55.979488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:46.630686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.985276+0000","last_change":"2024-04-25T12:05:03.663558+0000","last_active":"2024-04-25T12:05:55.985276+0000","last_peered":"2024-04-25T12:05:55.985276+0000","last_clean":"2024-04-25T12:05:55.985276+0000","last_became_active":"2024-04-25T12:05:03.661650+0000","last_became_peered":"2024-04-25T12:05:03.661650+0000","last_unstale":"2024-04-25T12:05:55.985276+0000","last_undegraded":"2024-04-25T12:05:55.985276+0000","last_fullsized":"2024-04-25T12:05:55.985276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:27.599992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.989021+0000","last_change":"2024-04-25T12:05:55.989021+0000","last_active":"2024-04-25T12:05:55.989021+0000","last_peered":"2024-04-25T12:05:55.989021+0000","last_clean":"2024-04-25T12:05:55.989021+0000","last_became_active":"2024-04-25T12:05:55.988246+0000","last_became_peered":"2024-04-25T12:05:55.988246+0000","last_unstale":"2024-04-25T12:05:55.989021+0000","last_undegraded":"2024-04-25T12:05:55.989021+0000","last_fullsized":"2024-04-25T12:05:55.989021+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:34.237743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984111+0000","last_change":"2024-04-25T12:05:03.662922+0000","last_active":"2024-04-25T12:05:55.984111+0000","last_peered":"2024-04-25T12:05:55.984111+0000","last_clean":"2024-04-25T12:05:55.984111+0000","last_became_active":"2024-04-25T12:05:03.662584+0000","last_became_peered":"2024-04-25T12:05:03.662584+0000","last_unstale":"2024-04-25T12:05:55.984111+0000","last_undegraded":"2024-04-25T12:05:55.984111+0000","last_fullsized":"2024-04-25T12:05:55.984111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:41.664030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983664+0000","last_change":"2024-04-25T12:05:04.672018+0000","last_active":"2024-04-25T12:05:55.983664+0000","last_peered":"2024-04-25T12:05:55.983664+0000","last_clean":"2024-04-25T12:05:55.983664+0000","last_became_active":"2024-04-25T12:05:04.671670+0000","last_became_peered":"2024-04-25T12:05:04.671670+0000","last_unstale":"2024-04-25T12:05:55.983664+0000","last_undegraded":"2024-04-25T12:05:55.983664+0000","last_fullsized":"2024-04-25T12:05:55.983664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:45.005941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":100,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:21.308073+0000","last_change":"2024-04-25T12:05:03.656826+0000","last_active":"2024-04-25T12:06:21.308073+0000","last_peered":"2024-04-25T12:06:21.308073+0000","last_clean":"2024-04-25T12:06:21.308073+0000","last_became_active":"2024-04-25T12:05:03.656477+0000","last_became_peered":"2024-04-25T12:05:03.656477+0000","last_unstale":"2024-04-25T12:06:21.308073+0000","last_undegraded":"2024-04-25T12:06:21.308073+0000","last_fullsized":"2024-04-25T12:06:21.308073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:38.111569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983325+0000","last_change":"2024-04-25T12:05:04.671771+0000","last_active":"2024-04-25T12:05:55.983325+0000","last_peered":"2024-04-25T12:05:55.983325+0000","last_clean":"2024-04-25T12:05:55.983325+0000","last_became_active":"2024-04-25T12:05:04.671424+0000","last_became_peered":"2024-04-25T12:05:04.671424+0000","last_unstale":"2024-04-25T12:05:55.983325+0000","last_undegraded":"2024-04-25T12:05:55.983325+0000","last_fullsized":"2024-04-25T12:05:55.983325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:51.327337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979202+0000","last_change":"2024-04-25T12:05:03.655884+0000","last_active":"2024-04-25T12:05:55.979202+0000","last_peered":"2024-04-25T12:05:55.979202+0000","last_clean":"2024-04-25T12:05:55.979202+0000","last_became_active":"2024-04-25T12:05:03.655437+0000","last_became_peered":"2024-04-25T12:05:03.655437+0000","last_unstale":"2024-04-25T12:05:55.979202+0000","last_undegraded":"2024-04-25T12:05:55.979202+0000","last_fullsized":"2024-04-25T12:05:55.979202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:24.497861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984027+0000","last_change":"2024-04-25T12:05:04.674785+0000","last_active":"2024-04-25T12:05:55.984027+0000","last_peered":"2024-04-25T12:05:55.984027+0000","last_clean":"2024-04-25T12:05:55.984027+0000","last_became_active":"2024-04-25T12:05:04.674469+0000","last_became_peered":"2024-04-25T12:05:04.674469+0000","last_unstale":"2024-04-25T12:05:55.984027+0000","last_undegraded":"2024-04-25T12:05:55.984027+0000","last_fullsized":"2024-04-25T12:05:55.984027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:28.616188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979699+0000","last_change":"2024-04-25T12:05:03.653954+0000","last_active":"2024-04-25T12:05:55.979699+0000","last_peered":"2024-04-25T12:05:55.979699+0000","last_clean":"2024-04-25T12:05:55.979699+0000","last_became_active":"2024-04-25T12:05:03.653583+0000","last_became_peered":"2024-04-25T12:05:03.653583+0000","last_unstale":"2024-04-25T12:05:55.979699+0000","last_undegraded":"2024-04-25T12:05:55.979699+0000","last_fullsized":"2024-04-25T12:05:55.979699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:58.007741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981248+0000","last_change":"2024-04-25T12:05:04.669395+0000","last_active":"2024-04-25T12:05:55.981248+0000","last_peered":"2024-04-25T12:05:55.981248+0000","last_clean":"2024-04-25T12:05:55.981248+0000","last_became_active":"2024-04-25T12:05:04.669058+0000","last_became_peered":"2024-04-25T12:05:04.669058+0000","last_unstale":"2024-04-25T12:05:55.981248+0000","last_undegraded":"2024-04-25T12:05:55.981248+0000","last_fullsized":"2024-04-25T12:05:55.981248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:03.195889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981946+0000","last_change":"2024-04-25T12:05:03.658899+0000","last_active":"2024-04-25T12:05:55.981946+0000","last_peered":"2024-04-25T12:05:55.981946+0000","last_clean":"2024-04-25T12:05:55.981946+0000","last_became_active":"2024-04-25T12:05:03.658521+0000","last_became_peered":"2024-04-25T12:05:03.658521+0000","last_unstale":"2024-04-25T12:05:55.981946+0000","last_undegraded":"2024-04-25T12:05:55.981946+0000","last_fullsized":"2024-04-25T12:05:55.981946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:21.296484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982466+0000","last_change":"2024-04-25T12:05:04.665958+0000","last_active":"2024-04-25T12:05:55.982466+0000","last_peered":"2024-04-25T12:05:55.982466+0000","last_clean":"2024-04-25T12:05:55.982466+0000","last_became_active":"2024-04-25T12:05:04.665619+0000","last_became_peered":"2024-04-25T12:05:04.665619+0000","last_unstale":"2024-04-25T12:05:55.982466+0000","last_undegraded":"2024-04-25T12:05:55.982466+0000","last_fullsized":"2024-04-25T12:05:55.982466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:46.837053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:23.026441+0000","last_change":"2024-04-25T12:05:03.661021+0000","last_active":"2024-04-25T12:06:23.026441+0000","last_peered":"2024-04-25T12:06:23.026441+0000","last_clean":"2024-04-25T12:06:23.026441+0000","last_became_active":"2024-04-25T12:05:03.660690+0000","last_became_peered":"2024-04-25T12:05:03.660690+0000","last_unstale":"2024-04-25T12:06:23.026441+0000","last_undegraded":"2024-04-25T12:06:23.026441+0000","last_fullsized":"2024-04-25T12:06:23.026441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:00.645287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982172+0000","last_change":"2024-04-25T12:05:04.669496+0000","last_active":"2024-04-25T12:05:55.982172+0000","last_peered":"2024-04-25T12:05:55.982172+0000","last_clean":"2024-04-25T12:05:55.982172+0000","last_became_active":"2024-04-25T12:05:04.669173+0000","last_became_peered":"2024-04-25T12:05:04.669173+0000","last_unstale":"2024-04-25T12:05:55.982172+0000","last_undegraded":"2024-04-25T12:05:55.982172+0000","last_fullsized":"2024-04-25T12:05:55.982172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:30.527192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983609+0000","last_change":"2024-04-25T12:05:04.673622+0000","last_active":"2024-04-25T12:05:55.983609+0000","last_peered":"2024-04-25T12:05:55.983609+0000","last_clean":"2024-04-25T12:05:55.983609+0000","last_became_active":"2024-04-25T12:05:04.673245+0000","last_became_peered":"2024-04-25T12:05:04.673245+0000","last_unstale":"2024-04-25T12:05:55.983609+0000","last_undegraded":"2024-04-25T12:05:55.983609+0000","last_fullsized":"2024-04-25T12:05:55.983609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:59.180387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:21.307091+0000","last_change":"2024-04-25T12:05:03.665419+0000","last_active":"2024-04-25T12:06:21.307091+0000","last_peered":"2024-04-25T12:06:21.307091+0000","last_clean":"2024-04-25T12:06:21.307091+0000","last_became_active":"2024-04-25T12:05:03.664963+0000","last_became_peered":"2024-04-25T12:05:03.664963+0000","last_unstale":"2024-04-25T12:06:21.307091+0000","last_undegraded":"2024-04-25T12:06:21.307091+0000","last_fullsized":"2024-04-25T12:06:21.307091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:04.074104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987464+0000","last_change":"2024-04-25T12:05:04.674257+0000","last_active":"2024-04-25T12:05:55.987464+0000","last_peered":"2024-04-25T12:05:55.987464+0000","last_clean":"2024-04-25T12:05:55.987464+0000","last_became_active":"2024-04-25T12:05:04.669726+0000","last_became_peered":"2024-04-25T12:05:04.669726+0000","last_unstale":"2024-04-25T12:05:55.987464+0000","last_undegraded":"2024-04-25T12:05:55.987464+0000","last_fullsized":"2024-04-25T12:05:55.987464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:03.304088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.980083+0000","last_change":"2024-04-25T12:05:03.652149+0000","last_active":"2024-04-25T12:05:55.980083+0000","last_peered":"2024-04-25T12:05:55.980083+0000","last_clean":"2024-04-25T12:05:55.980083+0000","last_became_active":"2024-04-25T12:05:03.650216+0000","last_became_peered":"2024-04-25T12:05:03.650216+0000","last_unstale":"2024-04-25T12:05:55.980083+0000","last_undegraded":"2024-04-25T12:05:55.980083+0000","last_fullsized":"2024-04-25T12:05:55.980083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:00.255226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983450+0000","last_change":"2024-04-25T12:03:51.170356+0000","last_active":"2024-04-25T12:05:55.983450+0000","last_peered":"2024-04-25T12:05:55.983450+0000","last_clean":"2024-04-25T12:05:55.983450+0000","last_became_active":"2024-04-25T12:03:51.169795+0000","last_became_peered":"2024-04-25T12:03:51.169795+0000","last_unstale":"2024-04-25T12:05:55.983450+0000","last_undegraded":"2024-04-25T12:05:55.983450+0000","last_fullsized":"2024-04-25T12:05:55.983450+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:00:21.810858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:00:21.810858+0000","last_clean_scrub_stamp":"2024-04-25T12:00:21.810858+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:34.246289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981579+0000","last_change":"2024-04-25T12:05:04.671882+0000","last_active":"2024-04-25T12:05:55.981579+0000","last_peered":"2024-04-25T12:05:55.981579+0000","last_clean":"2024-04-25T12:05:55.981579+0000","last_became_active":"2024-04-25T12:05:04.671408+0000","last_became_peered":"2024-04-25T12:05:04.671408+0000","last_unstale":"2024-04-25T12:05:55.981579+0000","last_undegraded":"2024-04-25T12:05:55.981579+0000","last_fullsized":"2024-04-25T12:05:55.981579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:36.465054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983662+0000","last_change":"2024-04-25T12:05:03.659082+0000","last_active":"2024-04-25T12:05:55.983662+0000","last_peered":"2024-04-25T12:05:55.983662+0000","last_clean":"2024-04-25T12:05:55.983662+0000","last_became_active":"2024-04-25T12:05:03.658677+0000","last_became_peered":"2024-04-25T12:05:03.658677+0000","last_unstale":"2024-04-25T12:05:55.983662+0000","last_undegraded":"2024-04-25T12:05:55.983662+0000","last_fullsized":"2024-04-25T12:05:55.983662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:20.938394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979547+0000","last_change":"2024-04-25T12:05:03.661172+0000","last_active":"2024-04-25T12:05:55.979547+0000","last_peered":"2024-04-25T12:05:55.979547+0000","last_clean":"2024-04-25T12:05:55.979547+0000","last_became_active":"2024-04-25T12:05:03.660799+0000","last_became_peered":"2024-04-25T12:05:03.660799+0000","last_unstale":"2024-04-25T12:05:55.979547+0000","last_undegraded":"2024-04-25T12:05:55.979547+0000","last_fullsized":"2024-04-25T12:05:55.979547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:01.889714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982555+0000","last_change":"2024-04-25T12:05:04.662203+0000","last_active":"2024-04-25T12:05:55.982555+0000","last_peered":"2024-04-25T12:05:55.982555+0000","last_clean":"2024-04-25T12:05:55.982555+0000","last_became_active":"2024-04-25T12:05:04.661816+0000","last_became_peered":"2024-04-25T12:05:04.661816+0000","last_unstale":"2024-04-25T12:05:55.982555+0000","last_undegraded":"2024-04-25T12:05:55.982555+0000","last_fullsized":"2024-04-25T12:05:55.982555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:06.656668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981384+0000","last_change":"2024-04-25T12:05:03.664741+0000","last_active":"2024-04-25T12:05:55.981384+0000","last_peered":"2024-04-25T12:05:55.981384+0000","last_clean":"2024-04-25T12:05:55.981384+0000","last_became_active":"2024-04-25T12:05:03.664373+0000","last_became_peered":"2024-04-25T12:05:03.664373+0000","last_unstale":"2024-04-25T12:05:55.981384+0000","last_undegraded":"2024-04-25T12:05:55.981384+0000","last_fullsized":"2024-04-25T12:05:55.981384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:03.540606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987448+0000","last_change":"2024-04-25T12:05:04.677748+0000","last_active":"2024-04-25T12:05:55.987448+0000","last_peered":"2024-04-25T12:05:55.987448+0000","last_clean":"2024-04-25T12:05:55.987448+0000","last_became_active":"2024-04-25T12:05:04.674003+0000","last_became_peered":"2024-04-25T12:05:04.674003+0000","last_unstale":"2024-04-25T12:05:55.987448+0000","last_undegraded":"2024-04-25T12:05:55.987448+0000","last_fullsized":"2024-04-25T12:05:55.987448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:24.031333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.980089+0000","last_change":"2024-04-25T12:05:03.666295+0000","last_active":"2024-04-25T12:05:55.980089+0000","last_peered":"2024-04-25T12:05:55.980089+0000","last_clean":"2024-04-25T12:05:55.980089+0000","last_became_active":"2024-04-25T12:05:03.665910+0000","last_became_peered":"2024-04-25T12:05:03.665910+0000","last_unstale":"2024-04-25T12:05:55.980089+0000","last_undegraded":"2024-04-25T12:05:55.980089+0000","last_fullsized":"2024-04-25T12:05:55.980089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:13.028799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.988948+0000","last_change":"2024-04-25T12:05:55.988948+0000","last_active":"2024-04-25T12:05:55.988948+0000","last_peered":"2024-04-25T12:05:55.988948+0000","last_clean":"2024-04-25T12:05:55.988948+0000","last_became_active":"2024-04-25T12:05:55.988373+0000","last_became_peered":"2024-04-25T12:05:55.988373+0000","last_unstale":"2024-04-25T12:05:55.988948+0000","last_undegraded":"2024-04-25T12:05:55.988948+0000","last_fullsized":"2024-04-25T12:05:55.988948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:29.663313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987535+0000","last_change":"2024-04-25T12:05:03.663799+0000","last_active":"2024-04-25T12:05:55.987535+0000","last_peered":"2024-04-25T12:05:55.987535+0000","last_clean":"2024-04-25T12:05:55.987535+0000","last_became_active":"2024-04-25T12:05:03.663501+0000","last_became_peered":"2024-04-25T12:05:03.663501+0000","last_unstale":"2024-04-25T12:05:55.987535+0000","last_undegraded":"2024-04-25T12:05:55.987535+0000","last_fullsized":"2024-04-25T12:05:55.987535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:34.402036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979202+0000","last_change":"2024-04-25T12:05:04.662594+0000","last_active":"2024-04-25T12:05:55.979202+0000","last_peered":"2024-04-25T12:05:55.979202+0000","last_clean":"2024-04-25T12:05:55.979202+0000","last_became_active":"2024-04-25T12:05:04.662179+0000","last_became_peered":"2024-04-25T12:05:04.662179+0000","last_unstale":"2024-04-25T12:05:55.979202+0000","last_undegraded":"2024-04-25T12:05:55.979202+0000","last_fullsized":"2024-04-25T12:05:55.979202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:58.709837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987161+0000","last_change":"2024-04-25T12:05:03.659416+0000","last_active":"2024-04-25T12:05:55.987161+0000","last_peered":"2024-04-25T12:05:55.987161+0000","last_clean":"2024-04-25T12:05:55.987161+0000","last_became_active":"2024-04-25T12:05:03.658416+0000","last_became_peered":"2024-04-25T12:05:03.658416+0000","last_unstale":"2024-04-25T12:05:55.987161+0000","last_undegraded":"2024-04-25T12:05:55.987161+0000","last_fullsized":"2024-04-25T12:05:55.987161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:01.174438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.980963+0000","last_change":"2024-04-25T12:05:04.670890+0000","last_active":"2024-04-25T12:05:55.980963+0000","last_peered":"2024-04-25T12:05:55.980963+0000","last_clean":"2024-04-25T12:05:55.980963+0000","last_became_active":"2024-04-25T12:05:04.670581+0000","last_became_peered":"2024-04-25T12:05:04.670581+0000","last_unstale":"2024-04-25T12:05:55.980963+0000","last_undegraded":"2024-04-25T12:05:55.980963+0000","last_fullsized":"2024-04-25T12:05:55.980963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.527303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984360+0000","last_change":"2024-04-25T12:05:03.662561+0000","last_active":"2024-04-25T12:05:55.984360+0000","last_peered":"2024-04-25T12:05:55.984360+0000","last_clean":"2024-04-25T12:05:55.984360+0000","last_became_active":"2024-04-25T12:05:03.662155+0000","last_became_peered":"2024-04-25T12:05:03.662155+0000","last_unstale":"2024-04-25T12:05:55.984360+0000","last_undegraded":"2024-04-25T12:05:55.984360+0000","last_fullsized":"2024-04-25T12:05:55.984360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:02.130271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987327+0000","last_change":"2024-04-25T12:05:04.669905+0000","last_active":"2024-04-25T12:05:55.987327+0000","last_peered":"2024-04-25T12:05:55.987327+0000","last_clean":"2024-04-25T12:05:55.987327+0000","last_became_active":"2024-04-25T12:05:04.669322+0000","last_became_peered":"2024-04-25T12:05:04.669322+0000","last_unstale":"2024-04-25T12:05:55.987327+0000","last_undegraded":"2024-04-25T12:05:55.987327+0000","last_fullsized":"2024-04-25T12:05:55.987327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.705643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.979364+0000","last_change":"2024-04-25T12:05:03.657923+0000","last_active":"2024-04-25T12:05:55.979364+0000","last_peered":"2024-04-25T12:05:55.979364+0000","last_clean":"2024-04-25T12:05:55.979364+0000","last_became_active":"2024-04-25T12:05:03.657332+0000","last_became_peered":"2024-04-25T12:05:03.657332+0000","last_unstale":"2024-04-25T12:05:55.979364+0000","last_undegraded":"2024-04-25T12:05:55.979364+0000","last_fullsized":"2024-04-25T12:05:55.979364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:54.521822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983652+0000","last_change":"2024-04-25T12:05:04.670193+0000","last_active":"2024-04-25T12:05:55.983652+0000","last_peered":"2024-04-25T12:05:55.983652+0000","last_clean":"2024-04-25T12:05:55.983652+0000","last_became_active":"2024-04-25T12:05:04.669720+0000","last_became_peered":"2024-04-25T12:05:04.669720+0000","last_unstale":"2024-04-25T12:05:55.983652+0000","last_undegraded":"2024-04-25T12:05:55.983652+0000","last_fullsized":"2024-04-25T12:05:55.983652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:26.080914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:06:19.286776+0000","last_change":"2024-04-25T12:05:03.666064+0000","last_active":"2024-04-25T12:06:19.286776+0000","last_peered":"2024-04-25T12:06:19.286776+0000","last_clean":"2024-04-25T12:06:19.286776+0000","last_became_active":"2024-04-25T12:05:03.665749+0000","last_became_peered":"2024-04-25T12:05:03.665749+0000","last_unstale":"2024-04-25T12:06:19.286776+0000","last_undegraded":"2024-04-25T12:06:19.286776+0000","last_fullsized":"2024-04-25T12:06:19.286776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:21.607549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983657+0000","last_change":"2024-04-25T12:05:04.660953+0000","last_active":"2024-04-25T12:05:55.983657+0000","last_peered":"2024-04-25T12:05:55.983657+0000","last_clean":"2024-04-25T12:05:55.983657+0000","last_became_active":"2024-04-25T12:05:04.660577+0000","last_became_peered":"2024-04-25T12:05:04.660577+0000","last_unstale":"2024-04-25T12:05:55.983657+0000","last_undegraded":"2024-04-25T12:05:55.983657+0000","last_fullsized":"2024-04-25T12:05:55.983657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:35.728839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984456+0000","last_change":"2024-04-25T12:05:03.664454+0000","last_active":"2024-04-25T12:05:55.984456+0000","last_peered":"2024-04-25T12:05:55.984456+0000","last_clean":"2024-04-25T12:05:55.984456+0000","last_became_active":"2024-04-25T12:05:03.664064+0000","last_became_peered":"2024-04-25T12:05:03.664064+0000","last_unstale":"2024-04-25T12:05:55.984456+0000","last_undegraded":"2024-04-25T12:05:55.984456+0000","last_fullsized":"2024-04-25T12:05:55.984456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:28.146713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983622+0000","last_change":"2024-04-25T12:05:04.668385+0000","last_active":"2024-04-25T12:05:55.983622+0000","last_peered":"2024-04-25T12:05:55.983622+0000","last_clean":"2024-04-25T12:05:55.983622+0000","last_became_active":"2024-04-25T12:05:04.667995+0000","last_became_peered":"2024-04-25T12:05:04.667995+0000","last_unstale":"2024-04-25T12:05:55.983622+0000","last_undegraded":"2024-04-25T12:05:55.983622+0000","last_fullsized":"2024-04-25T12:05:55.983622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:39.938151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.994786+0000","last_change":"2024-04-25T12:05:55.994786+0000","last_active":"2024-04-25T12:05:55.994786+0000","last_peered":"2024-04-25T12:05:55.994786+0000","last_clean":"2024-04-25T12:05:55.994786+0000","last_became_active":"2024-04-25T12:05:55.990990+0000","last_became_peered":"2024-04-25T12:05:55.990990+0000","last_unstale":"2024-04-25T12:05:55.994786+0000","last_undegraded":"2024-04-25T12:05:55.994786+0000","last_fullsized":"2024-04-25T12:05:55.994786+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:02.636027+0000","last_clean_scrub_stamp":"2024-04-25T12:05:02.636027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:02.526236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983767+0000","last_change":"2024-04-25T12:05:04.662068+0000","last_active":"2024-04-25T12:05:55.983767+0000","last_peered":"2024-04-25T12:05:55.983767+0000","last_clean":"2024-04-25T12:05:55.983767+0000","last_became_active":"2024-04-25T12:05:04.661694+0000","last_became_peered":"2024-04-25T12:05:04.661694+0000","last_unstale":"2024-04-25T12:05:55.983767+0000","last_undegraded":"2024-04-25T12:05:55.983767+0000","last_fullsized":"2024-04-25T12:05:55.983767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:45.200872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981038+0000","last_change":"2024-04-25T12:05:04.662320+0000","last_active":"2024-04-25T12:05:55.981038+0000","last_peered":"2024-04-25T12:05:55.981038+0000","last_clean":"2024-04-25T12:05:55.981038+0000","last_became_active":"2024-04-25T12:05:04.661992+0000","last_became_peered":"2024-04-25T12:05:04.661992+0000","last_unstale":"2024-04-25T12:05:55.981038+0000","last_undegraded":"2024-04-25T12:05:55.981038+0000","last_fullsized":"2024-04-25T12:05:55.981038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:25.898506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984378+0000","last_change":"2024-04-25T12:05:04.671534+0000","last_active":"2024-04-25T12:05:55.984378+0000","last_peered":"2024-04-25T12:05:55.984378+0000","last_clean":"2024-04-25T12:05:55.984378+0000","last_became_active":"2024-04-25T12:05:04.671169+0000","last_became_peered":"2024-04-25T12:05:04.671169+0000","last_unstale":"2024-04-25T12:05:55.984378+0000","last_undegraded":"2024-04-25T12:05:55.984378+0000","last_fullsized":"2024-04-25T12:05:55.984378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:49.972884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.984217+0000","last_change":"2024-04-25T12:05:04.668832+0000","last_active":"2024-04-25T12:05:55.984217+0000","last_peered":"2024-04-25T12:05:55.984217+0000","last_clean":"2024-04-25T12:05:55.984217+0000","last_became_active":"2024-04-25T12:05:04.668459+0000","last_became_peered":"2024-04-25T12:05:04.668459+0000","last_unstale":"2024-04-25T12:05:55.984217+0000","last_undegraded":"2024-04-25T12:05:55.984217+0000","last_fullsized":"2024-04-25T12:05:55.984217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:58.509505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982220+0000","last_change":"2024-04-25T12:05:04.671393+0000","last_active":"2024-04-25T12:05:55.982220+0000","last_peered":"2024-04-25T12:05:55.982220+0000","last_clean":"2024-04-25T12:05:55.982220+0000","last_became_active":"2024-04-25T12:05:04.671041+0000","last_became_peered":"2024-04-25T12:05:04.671041+0000","last_unstale":"2024-04-25T12:05:55.982220+0000","last_undegraded":"2024-04-25T12:05:55.982220+0000","last_fullsized":"2024-04-25T12:05:55.982220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:29.428573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.986680+0000","last_change":"2024-04-25T12:05:55.986680+0000","last_active":"2024-04-25T12:05:55.986680+0000","last_peered":"2024-04-25T12:05:55.986680+0000","last_clean":"2024-04-25T12:05:55.986680+0000","last_became_active":"2024-04-25T12:05:55.986252+0000","last_became_peered":"2024-04-25T12:05:55.986252+0000","last_unstale":"2024-04-25T12:05:55.986680+0000","last_undegraded":"2024-04-25T12:05:55.986680+0000","last_fullsized":"2024-04-25T12:05:55.986680+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:13.570601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981722+0000","last_change":"2024-04-25T12:05:04.664852+0000","last_active":"2024-04-25T12:05:55.981722+0000","last_peered":"2024-04-25T12:05:55.981722+0000","last_clean":"2024-04-25T12:05:55.981722+0000","last_became_active":"2024-04-25T12:05:04.663269+0000","last_became_peered":"2024-04-25T12:05:04.663269+0000","last_unstale":"2024-04-25T12:05:55.981722+0000","last_undegraded":"2024-04-25T12:05:55.981722+0000","last_fullsized":"2024-04-25T12:05:55.981722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:57.862378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.985625+0000","last_change":"2024-04-25T12:05:04.666332+0000","last_active":"2024-04-25T12:05:55.985625+0000","last_peered":"2024-04-25T12:05:55.985625+0000","last_clean":"2024-04-25T12:05:55.985625+0000","last_became_active":"2024-04-25T12:05:04.665777+0000","last_became_peered":"2024-04-25T12:05:04.665777+0000","last_unstale":"2024-04-25T12:05:55.985625+0000","last_undegraded":"2024-04-25T12:05:55.985625+0000","last_fullsized":"2024-04-25T12:05:55.985625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:53.912782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987455+0000","last_change":"2024-04-25T12:05:04.678238+0000","last_active":"2024-04-25T12:05:55.987455+0000","last_peered":"2024-04-25T12:05:55.987455+0000","last_clean":"2024-04-25T12:05:55.987455+0000","last_became_active":"2024-04-25T12:05:04.677660+0000","last_became_peered":"2024-04-25T12:05:04.677660+0000","last_unstale":"2024-04-25T12:05:55.987455+0000","last_undegraded":"2024-04-25T12:05:55.987455+0000","last_fullsized":"2024-04-25T12:05:55.987455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:02.916486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.983490+0000","last_change":"2024-04-25T12:05:04.666533+0000","last_active":"2024-04-25T12:05:55.983490+0000","last_peered":"2024-04-25T12:05:55.983490+0000","last_clean":"2024-04-25T12:05:55.983490+0000","last_became_active":"2024-04-25T12:05:04.666184+0000","last_became_peered":"2024-04-25T12:05:04.666184+0000","last_unstale":"2024-04-25T12:05:55.983490+0000","last_undegraded":"2024-04-25T12:05:55.983490+0000","last_fullsized":"2024-04-25T12:05:55.983490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:13.089285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987209+0000","last_change":"2024-04-25T12:05:04.672880+0000","last_active":"2024-04-25T12:05:55.987209+0000","last_peered":"2024-04-25T12:05:55.987209+0000","last_clean":"2024-04-25T12:05:55.987209+0000","last_became_active":"2024-04-25T12:05:04.669640+0000","last_became_peered":"2024-04-25T12:05:04.669640+0000","last_unstale":"2024-04-25T12:05:55.987209+0000","last_undegraded":"2024-04-25T12:05:55.987209+0000","last_fullsized":"2024-04-25T12:05:55.987209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:52.293914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.987751+0000","last_change":"2024-04-25T12:05:04.664274+0000","last_active":"2024-04-25T12:05:55.987751+0000","last_peered":"2024-04-25T12:05:55.987751+0000","last_clean":"2024-04-25T12:05:55.987751+0000","last_became_active":"2024-04-25T12:05:04.663715+0000","last_became_peered":"2024-04-25T12:05:04.663715+0000","last_unstale":"2024-04-25T12:05:55.987751+0000","last_undegraded":"2024-04-25T12:05:55.987751+0000","last_fullsized":"2024-04-25T12:05:55.987751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:28.298147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.982780+0000","last_change":"2024-04-25T12:05:04.669974+0000","last_active":"2024-04-25T12:05:55.982780+0000","last_peered":"2024-04-25T12:05:55.982780+0000","last_clean":"2024-04-25T12:05:55.982780+0000","last_became_active":"2024-04-25T12:05:04.669531+0000","last_became_peered":"2024-04-25T12:05:04.669531+0000","last_unstale":"2024-04-25T12:05:55.982780+0000","last_undegraded":"2024-04-25T12:05:55.982780+0000","last_fullsized":"2024-04-25T12:05:55.982780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:51.579582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T12:05:55.981072+0000","last_change":"2024-04-25T12:05:04.670203+0000","last_active":"2024-04-25T12:05:55.981072+0000","last_peered":"2024-04-25T12:05:55.981072+0000","last_clean":"2024-04-25T12:05:55.981072+0000","last_became_active":"2024-04-25T12:05:04.669865+0000","last_became_peered":"2024-04-25T12:05:04.669865+0000","last_unstale":"2024-04-25T12:05:55.981072+0000","last_undegraded":"2024-04-25T12:05:55.981072+0000","last_fullsized":"2024-04-25T12:05:55.981072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T12:05:03.637777+0000","last_clean_scrub_stamp":"2024-04-25T12:05:03.637777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:19.860002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281216,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67859,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":701,"num_read_kb":446,"num_write":236,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1294,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":203577,"data_compressed":5586,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":457611,"data_compressed":5773,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1101824,"data_stored":1109455,"data_compressed":43402,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":456919,"data_compressed":5773,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":456625,"data_compressed":5773,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1049815,"data_compressed":41540,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":456636,"data_compressed":5773,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":519267,"data_compressed":7635,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":457975,"data_compressed":5773,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":518815,"data_compressed":7635,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":456461,"data_compressed":5773,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456888,"data_compressed":5773,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":1048480,"data_compressed":41540,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34127,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34127,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34127,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35767,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63470,"data_compressed":1862,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63018,"data_compressed":1862,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63173,"data_compressed":1862,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T12:06:27.611 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T12:06:27.611 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T12:06:27.611 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T12:06:27.611 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T12:06:27.864 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:06:28.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:28 smithi150 ceph-mon[32250]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:06:28.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:28 smithi150 ceph-mon[32250]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T12:06:29.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:28 smithi019 ceph-mon[29213]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:06:29.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:28 smithi019 ceph-mon[29213]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T12:06:29.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:28 smithi169 ceph-mon[32710]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T12:06:29.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:28 smithi169 ceph-mon[32710]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-25T12:06:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.618+0000 7fed8b581700 1 -- 172.21.15.19:0/3952036106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed84101930 msgr2=0x7fed84101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.618+0000 7fed8b581700 1 --2- 172.21.15.19:0/3952036106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed84101930 0x7fed84101db0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fed74009990 tx=0x7fed74009ca0 comp rx=0 tx=0).stop 2024-04-25T12:06:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.618+0000 7fed8b581700 1 -- 172.21.15.19:0/3952036106 shutdown_connections 2024-04-25T12:06:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.618+0000 7fed8b581700 1 --2- 172.21.15.19:0/3952036106 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed841022f0 0x7fed8410e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.618+0000 7fed8b581700 1 --2- 172.21.15.19:0/3952036106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed84107980 0x7fed84107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.618+0000 7fed8b581700 1 --2- 172.21.15.19:0/3952036106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed84101930 0x7fed84101db0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.618+0000 7fed8b581700 1 -- 172.21.15.19:0/3952036106 >> 172.21.15.19:0/3952036106 conn(0x7fed840788b0 msgr2=0x7fed84078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.619+0000 7fed8b581700 1 -- 172.21.15.19:0/3952036106 shutdown_connections 2024-04-25T12:06:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.619+0000 7fed8b581700 1 -- 172.21.15.19:0/3952036106 wait complete. 2024-04-25T12:06:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.619+0000 7fed8b581700 1 Processor -- start 2024-04-25T12:06:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.619+0000 7fed8b581700 1 -- start start 2024-04-25T12:06:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.619+0000 7fed8b581700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed84101930 0x7fed841034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.619+0000 7fed8b581700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed841022f0 0x7fed84103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.619+0000 7fed8b581700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed84107980 0x7fed84103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.619+0000 7fed8b581700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed84111c30 con 0x7fed84107980 2024-04-25T12:06:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.619+0000 7fed8b581700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fed84111ab0 con 0x7fed841022f0 2024-04-25T12:06:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.619+0000 7fed8b581700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fed84111db0 con 0x7fed84101930 2024-04-25T12:06:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed89b1e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed84107980 0x7fed84103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed89b1e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed84107980 0x7fed84103f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40952/0 (socket says 172.21.15.19:40952) 2024-04-25T12:06:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed89b1e700 1 -- 172.21.15.19:0/2433680142 learned_addr learned my addr 172.21.15.19:0/2433680142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed8931d700 1 --2- 172.21.15.19:0/2433680142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed84101930 0x7fed841034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed88b1c700 1 --2- 172.21.15.19:0/2433680142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed841022f0 0x7fed84103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed89b1e700 1 -- 172.21.15.19:0/2433680142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed84101930 msgr2=0x7fed841034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed89b1e700 1 --2- 172.21.15.19:0/2433680142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed84101930 0x7fed841034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed89b1e700 1 -- 172.21.15.19:0/2433680142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed841022f0 msgr2=0x7fed84103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed89b1e700 1 --2- 172.21.15.19:0/2433680142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed841022f0 0x7fed84103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed89b1e700 1 -- 172.21.15.19:0/2433680142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed841a7740 con 0x7fed84107980 2024-04-25T12:06:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed89b1e700 1 --2- 172.21.15.19:0/2433680142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed84107980 0x7fed84103f60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fed7c007ee0 tx=0x7fed7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed7a7fc700 1 -- 172.21.15.19:0/2433680142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c004bb0 con 0x7fed84107980 2024-04-25T12:06:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed8b581700 1 -- 172.21.15.19:0/2433680142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed841a7a30 con 0x7fed84107980 2024-04-25T12:06:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed7a7fc700 1 -- 172.21.15.19:0/2433680142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fed7c004d50 con 0x7fed84107980 2024-04-25T12:06:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed7a7fc700 1 -- 172.21.15.19:0/2433680142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0059e0 con 0x7fed84107980 2024-04-25T12:06:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.620+0000 7fed8b581700 1 -- 172.21.15.19:0/2433680142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed841a7f40 con 0x7fed84107980 2024-04-25T12:06:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.621+0000 7fed8b581700 1 -- 172.21.15.19:0/2433680142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed84069730 con 0x7fed84107980 2024-04-25T12:06:29.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.622+0000 7fed7a7fc700 1 -- 172.21.15.19:0/2433680142 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fed7c01a020 con 0x7fed84107980 2024-04-25T12:06:29.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.623+0000 7fed7a7fc700 1 --2- 172.21.15.19:0/2433680142 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fed700a3c40 0x7fed700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:29.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.623+0000 7fed8931d700 1 --2- 172.21.15.19:0/2433680142 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fed700a3c40 0x7fed700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:29.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.623+0000 7fed7a7fc700 1 -- 172.21.15.19:0/2433680142 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fed7c0cfa80 con 0x7fed84107980 2024-04-25T12:06:29.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.623+0000 7fed8931d700 1 --2- 172.21.15.19:0/2433680142 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fed700a3c40 0x7fed700a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fed80005950 tx=0x7fed8000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:29.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.628+0000 7fed7a7fc700 1 -- 172.21.15.19:0/2433680142 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fed7c010070 con 0x7fed84107980 2024-04-25T12:06:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.886+0000 7fed8b581700 1 -- 172.21.15.19:0/2433680142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fed8402df80 con 0x7fed84107980 2024-04-25T12:06:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.886+0000 7fed7a7fc700 1 -- 172.21.15.19:0/2433680142 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fed7c09c440 con 0x7fed84107980 2024-04-25T12:06:29.886 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:29.886 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T12:06:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.888+0000 7fed8b581700 1 -- 172.21.15.19:0/2433680142 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fed700a3c40 msgr2=0x7fed700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.888+0000 7fed8b581700 1 --2- 172.21.15.19:0/2433680142 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fed700a3c40 0x7fed700a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fed80005950 tx=0x7fed8000b410 comp rx=0 tx=0).stop 2024-04-25T12:06:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.888+0000 7fed8b581700 1 -- 172.21.15.19:0/2433680142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed84107980 msgr2=0x7fed84103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.888+0000 7fed8b581700 1 --2- 172.21.15.19:0/2433680142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed84107980 0x7fed84103f60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fed7c007ee0 tx=0x7fed7c00d3b0 comp rx=0 tx=0).stop 2024-04-25T12:06:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.888+0000 7fed8b581700 1 -- 172.21.15.19:0/2433680142 shutdown_connections 2024-04-25T12:06:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.888+0000 7fed8b581700 1 --2- 172.21.15.19:0/2433680142 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fed700a3c40 0x7fed700a6100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.888+0000 7fed8b581700 1 --2- 172.21.15.19:0/2433680142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed84107980 0x7fed84103f60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.888+0000 7fed8b581700 1 --2- 172.21.15.19:0/2433680142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed84101930 0x7fed841034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.888+0000 7fed8b581700 1 --2- 172.21.15.19:0/2433680142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed841022f0 0x7fed84103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.888+0000 7fed8b581700 1 -- 172.21.15.19:0/2433680142 >> 172.21.15.19:0/2433680142 conn(0x7fed840788b0 msgr2=0x7fed8406b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.889+0000 7fed8b581700 1 -- 172.21.15.19:0/2433680142 shutdown_connections 2024-04-25T12:06:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:29.889+0000 7fed8b581700 1 -- 172.21.15.19:0/2433680142 wait complete. 2024-04-25T12:06:30.611 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T12:06:30.612 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:06:30.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.723+0000 7f33e988a640 1 -- 172.21.15.19:0/1258723036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33e4103b70 msgr2=0x7f33e410a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:30.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.723+0000 7f33e988a640 1 --2- 172.21.15.19:0/1258723036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33e4103b70 0x7f33e410a400 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009ca0 tx=0x7f33dc02a8c0 comp rx=0 tx=0).stop 2024-04-25T12:06:30.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.723+0000 7f33e988a640 1 -- 172.21.15.19:0/1258723036 shutdown_connections 2024-04-25T12:06:30.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.723+0000 7f33e988a640 1 --2- 172.21.15.19:0/1258723036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33e4103b70 0x7f33e410a400 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:30.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.723+0000 7f33e988a640 1 --2- 172.21.15.19:0/1258723036 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e41031b0 0x7f33e4103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:30.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.723+0000 7f33e988a640 1 --2- 172.21.15.19:0/1258723036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e4101fc0 0x7f33e41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:30.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.723+0000 7f33e988a640 1 -- 172.21.15.19:0/1258723036 >> 172.21.15.19:0/1258723036 conn(0x7f33e40fd550 msgr2=0x7f33e40ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:30.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.723+0000 7f33e988a640 1 -- 172.21.15.19:0/1258723036 shutdown_connections 2024-04-25T12:06:30.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.723+0000 7f33e988a640 1 -- 172.21.15.19:0/1258723036 wait complete. 2024-04-25T12:06:30.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.724+0000 7f33e988a640 1 Processor -- start 2024-04-25T12:06:30.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.724+0000 7f33e988a640 1 -- start start 2024-04-25T12:06:30.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.724+0000 7f33e988a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e4101fc0 0x7f33e4198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:30.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.724+0000 7f33e988a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33e41031b0 0x7f33e41987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:30.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.724+0000 7f33e988a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e4103b70 0x7f33e419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:30.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.724+0000 7f33e988a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33e410dc20 con 0x7f33e41031b0 2024-04-25T12:06:30.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.724+0000 7f33e988a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f33e410daa0 con 0x7f33e4101fc0 2024-04-25T12:06:30.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.724+0000 7f33e988a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f33e410dda0 con 0x7f33e4103b70 2024-04-25T12:06:30.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33e41031b0 0x7f33e41987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:30.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e27fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33e41031b0 0x7f33e41987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40970/0 (socket says 172.21.15.19:40970) 2024-04-25T12:06:30.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e27fc640 1 -- 172.21.15.19:0/51563641 learned_addr learned my addr 172.21.15.19:0/51563641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:30.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e2ffd640 1 --2- 172.21.15.19:0/51563641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e4101fc0 0x7f33e4198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:30.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e37fe640 1 --2- 172.21.15.19:0/51563641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e4103b70 0x7f33e419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:30.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e27fc640 1 -- 172.21.15.19:0/51563641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e4103b70 msgr2=0x7f33e419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:30.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e27fc640 1 --2- 172.21.15.19:0/51563641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e4103b70 0x7f33e419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:30.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e27fc640 1 -- 172.21.15.19:0/51563641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e4101fc0 msgr2=0x7f33e4198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:30.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e27fc640 1 --2- 172.21.15.19:0/51563641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e4101fc0 0x7f33e4198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:30.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e27fc640 1 -- 172.21.15.19:0/51563641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e419fe90 con 0x7f33e41031b0 2024-04-25T12:06:30.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e27fc640 1 --2- 172.21.15.19:0/51563641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33e41031b0 0x7f33e41987b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f33d800e9f0 tx=0x7f33d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:30.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e8888640 1 -- 172.21.15.19:0/51563641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d800cd80 con 0x7f33e41031b0 2024-04-25T12:06:30.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e8888640 1 -- 172.21.15.19:0/51563641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f33d80054f0 con 0x7f33e41031b0 2024-04-25T12:06:30.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e988a640 1 -- 172.21.15.19:0/51563641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e41a0180 con 0x7f33e41031b0 2024-04-25T12:06:30.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e8888640 1 -- 172.21.15.19:0/51563641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8010690 con 0x7f33e41031b0 2024-04-25T12:06:30.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.725+0000 7f33e988a640 1 -- 172.21.15.19:0/51563641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e41a0680 con 0x7f33e41031b0 2024-04-25T12:06:30.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.726+0000 7f33e988a640 1 -- 172.21.15.19:0/51563641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33e41092e0 con 0x7f33e41031b0 2024-04-25T12:06:30.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.727+0000 7f33e8888640 1 -- 172.21.15.19:0/51563641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f33d80046c0 con 0x7f33e41031b0 2024-04-25T12:06:30.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.728+0000 7f33e8888640 1 --2- 172.21.15.19:0/51563641 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f33cc0a3bb0 0x7f33cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:30.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.728+0000 7f33e2ffd640 1 --2- 172.21.15.19:0/51563641 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f33cc0a3bb0 0x7f33cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:30.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.728+0000 7f33e8888640 1 -- 172.21.15.19:0/51563641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f33d80d0b70 con 0x7f33e41031b0 2024-04-25T12:06:30.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.729+0000 7f33e2ffd640 1 --2- 172.21.15.19:0/51563641 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f33cc0a3bb0 0x7f33cc0a6070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f33d4005fd0 tx=0x7f33d4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:30.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.732+0000 7f33e8888640 1 -- 172.21.15.19:0/51563641 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f33d8053ff0 con 0x7f33e41031b0 2024-04-25T12:06:30.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:30 smithi150 ceph-mon[32250]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T12:06:30.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:30 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2433680142' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T12:06:30.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.919+0000 7f33e988a640 1 -- 172.21.15.19:0/51563641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33e402d810 con 0x7f33e41031b0 2024-04-25T12:06:30.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.921+0000 7f33e8888640 1 -- 172.21.15.19:0/51563641 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22731 (secure 0 0 0) 0x7f33d801f340 con 0x7f33e41031b0 2024-04-25T12:06:30.920 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:30.920 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":87,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:05:55.975412+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:05:01.658354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:05:02.969078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:06:30.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.923+0000 7f33e988a640 1 -- 172.21.15.19:0/51563641 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f33cc0a3bb0 msgr2=0x7f33cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:30.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.923+0000 7f33e988a640 1 --2- 172.21.15.19:0/51563641 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f33cc0a3bb0 0x7f33cc0a6070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f33d4005fd0 tx=0x7f33d4005850 comp rx=0 tx=0).stop 2024-04-25T12:06:30.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.923+0000 7f33e988a640 1 -- 172.21.15.19:0/51563641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33e41031b0 msgr2=0x7f33e41987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:30.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.923+0000 7f33e988a640 1 --2- 172.21.15.19:0/51563641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33e41031b0 0x7f33e41987b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f33d800e9f0 tx=0x7f33d800edf0 comp rx=0 tx=0).stop 2024-04-25T12:06:30.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.923+0000 7f33e988a640 1 -- 172.21.15.19:0/51563641 shutdown_connections 2024-04-25T12:06:30.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.923+0000 7f33e988a640 1 --2- 172.21.15.19:0/51563641 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f33cc0a3bb0 0x7f33cc0a6070 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:30.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.923+0000 7f33e988a640 1 --2- 172.21.15.19:0/51563641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33e4103b70 0x7f33e419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:30.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.923+0000 7f33e988a640 1 --2- 172.21.15.19:0/51563641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33e41031b0 0x7f33e41987b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:30.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.923+0000 7f33e988a640 1 --2- 172.21.15.19:0/51563641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33e4101fc0 0x7f33e4198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:30.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.923+0000 7f33e988a640 1 -- 172.21.15.19:0/51563641 >> 172.21.15.19:0/51563641 conn(0x7f33e40fd550 msgr2=0x7f33e41064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:30.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.923+0000 7f33e988a640 1 -- 172.21.15.19:0/51563641 shutdown_connections 2024-04-25T12:06:30.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:30.923+0000 7f33e988a640 1 -- 172.21.15.19:0/51563641 wait complete. 2024-04-25T12:06:30.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:30 smithi019 ceph-mon[29213]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T12:06:30.932 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:30 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2433680142' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T12:06:30.934 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:00:20.898809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:05:01.658354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:05:02.969078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T12:06:30.934 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:06:31.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.041+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/1200152001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb0102040 msgr2=0x7f4eb01024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.041+0000 7f4eb5ef5640 1 --2- 172.21.15.19:0/1200152001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb0102040 0x7f4eb01024c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009900 tx=0x7f4ea402a410 comp rx=0 tx=0).stop 2024-04-25T12:06:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.041+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/1200152001 shutdown_connections 2024-04-25T12:06:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.041+0000 7f4eb5ef5640 1 --2- 172.21.15.19:0/1200152001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb0102a00 0x7f4eb0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.041+0000 7f4eb5ef5640 1 --2- 172.21.15.19:0/1200152001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb0102040 0x7f4eb01024c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.041+0000 7f4eb5ef5640 1 --2- 172.21.15.19:0/1200152001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eb0100e60 0x7f4eb0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.041+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/1200152001 >> 172.21.15.19:0/1200152001 conn(0x7f4eb00fc440 msgr2=0x7f4eb00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.041+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/1200152001 shutdown_connections 2024-04-25T12:06:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.041+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/1200152001 wait complete. 2024-04-25T12:06:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eb5ef5640 1 Processor -- start 2024-04-25T12:06:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eb5ef5640 1 -- start start 2024-04-25T12:06:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eb5ef5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb0100e60 0x7f4eb0104db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eb5ef5640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb0102040 0x7f4eb01034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eb5ef5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eb0102a00 0x7f4eb01039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eb5ef5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb010cb30 con 0x7f4eb0100e60 2024-04-25T12:06:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eb5ef5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb010c9b0 con 0x7f4eb0102a00 2024-04-25T12:06:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eb5ef5640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb010ccb0 con 0x7f4eb0102040 2024-04-25T12:06:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eaf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb0100e60 0x7f4eb0104db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eaf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb0100e60 0x7f4eb0104db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40984/0 (socket says 172.21.15.19:40984) 2024-04-25T12:06:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eaf7fe640 1 -- 172.21.15.19:0/683085108 learned_addr learned my addr 172.21.15.19:0/683085108 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eaeffd640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb0102040 0x7f4eb01034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eaffff640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eb0102a00 0x7f4eb01039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eaf7fe640 1 -- 172.21.15.19:0/683085108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb0102040 msgr2=0x7f4eb01034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eaf7fe640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb0102040 0x7f4eb01034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eaf7fe640 1 -- 172.21.15.19:0/683085108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eb0102a00 msgr2=0x7f4eb01039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eaf7fe640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eb0102a00 0x7f4eb01039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eaf7fe640 1 -- 172.21.15.19:0/683085108 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb01041c0 con 0x7f4eb0100e60 2024-04-25T12:06:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.042+0000 7f4eaffff640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eb0102a00 0x7f4eb01039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.043+0000 7f4eaf7fe640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb0100e60 0x7f4eb0104db0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000c970 tx=0x7f4ea000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:31.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.043+0000 7f4eacff9640 1 -- 172.21.15.19:0/683085108 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0004220 con 0x7f4eb0100e60 2024-04-25T12:06:31.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.043+0000 7f4eacff9640 1 -- 172.21.15.19:0/683085108 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4ea00043c0 con 0x7f4eb0100e60 2024-04-25T12:06:31.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.043+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/683085108 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb01a23e0 con 0x7f4eb0100e60 2024-04-25T12:06:31.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.043+0000 7f4eacff9640 1 -- 172.21.15.19:0/683085108 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea000f450 con 0x7f4eb0100e60 2024-04-25T12:06:31.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.043+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/683085108 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb01a2950 con 0x7f4eb0100e60 2024-04-25T12:06:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.044+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/683085108 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb00691c0 con 0x7f4eb0100e60 2024-04-25T12:06:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.045+0000 7f4eacff9640 1 -- 172.21.15.19:0/683085108 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4ea0003730 con 0x7f4eb0100e60 2024-04-25T12:06:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.045+0000 7f4eacff9640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4e980a3c30 0x7f4e980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.045+0000 7f4eaeffd640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4e980a3c30 0x7f4e980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.045+0000 7f4eacff9640 1 -- 172.21.15.19:0/683085108 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f4ea00d00d0 con 0x7f4eb0100e60 2024-04-25T12:06:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.046+0000 7f4eaeffd640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4e980a3c30 0x7f4e980a60f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4000c00 tx=0x7f4ea4031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:31.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.050+0000 7f4eacff9640 1 -- 172.21.15.19:0/683085108 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ea0014030 con 0x7f4eb0100e60 2024-04-25T12:06:31.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:30 smithi169 ceph-mon[32710]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T12:06:31.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:30 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2433680142' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T12:06:31.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.236+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/683085108 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4eb01a32e0 con 0x7f4eb0100e60 2024-04-25T12:06:31.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.236+0000 7f4eacff9640 1 -- 172.21.15.19:0/683085108 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f4ea009ca90 con 0x7f4eb0100e60 2024-04-25T12:06:31.235 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T12:06:31.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.238+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/683085108 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4e980a3c30 msgr2=0x7f4e980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.238+0000 7f4eb5ef5640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4e980a3c30 0x7f4e980a60f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4000c00 tx=0x7f4ea4031510 comp rx=0 tx=0).stop 2024-04-25T12:06:31.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.238+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/683085108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb0100e60 msgr2=0x7f4eb0104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.238+0000 7f4eb5ef5640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb0100e60 0x7f4eb0104db0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000c970 tx=0x7f4ea000ce30 comp rx=0 tx=0).stop 2024-04-25T12:06:31.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.238+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/683085108 shutdown_connections 2024-04-25T12:06:31.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.238+0000 7f4eb5ef5640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4e980a3c30 0x7f4e980a60f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.238+0000 7f4eb5ef5640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eb0102a00 0x7f4eb01039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.238+0000 7f4eb5ef5640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4eb0102040 0x7f4eb01034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.238+0000 7f4eb5ef5640 1 --2- 172.21.15.19:0/683085108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb0100e60 0x7f4eb0104db0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.238+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/683085108 >> 172.21.15.19:0/683085108 conn(0x7f4eb00fc440 msgr2=0x7f4eb00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:31.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.238+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/683085108 shutdown_connections 2024-04-25T12:06:31.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.238+0000 7f4eb5ef5640 1 -- 172.21.15.19:0/683085108 wait complete. 2024-04-25T12:06:31.248 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T12:06:31.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.354+0000 7fdaba89a640 1 -- 172.21.15.19:0/4227945123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab4107c90 msgr2=0x7fdab410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.354+0000 7fdaba89a640 1 --2- 172.21.15.19:0/4227945123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab4107c90 0x7fdab410a0e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009ca0 tx=0x7fdaac02a8c0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.354+0000 7fdaba89a640 1 -- 172.21.15.19:0/4227945123 shutdown_connections 2024-04-25T12:06:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.354+0000 7fdaba89a640 1 --2- 172.21.15.19:0/4227945123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab4107c90 0x7fdab410a0e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.354+0000 7fdaba89a640 1 --2- 172.21.15.19:0/4227945123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4105360 0x7fdab4107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.354+0000 7fdaba89a640 1 --2- 172.21.15.19:0/4227945123 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab4102a30 0x7fdab4104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.354+0000 7fdaba89a640 1 -- 172.21.15.19:0/4227945123 >> 172.21.15.19:0/4227945123 conn(0x7fdab40fc420 msgr2=0x7fdab40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.354+0000 7fdaba89a640 1 -- 172.21.15.19:0/4227945123 shutdown_connections 2024-04-25T12:06:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.354+0000 7fdaba89a640 1 -- 172.21.15.19:0/4227945123 wait complete. 2024-04-25T12:06:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.355+0000 7fdaba89a640 1 Processor -- start 2024-04-25T12:06:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.355+0000 7fdaba89a640 1 -- start start 2024-04-25T12:06:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.355+0000 7fdaba89a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102a30 0x7fdab4198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.355+0000 7fdaba89a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab4105360 0x7fdab41987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.355+0000 7fdaba89a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab4107c90 0x7fdab419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.355+0000 7fdaba89a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdab410dc30 con 0x7fdab4107c90 2024-04-25T12:06:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.355+0000 7fdaba89a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdab410dab0 con 0x7fdab4102a30 2024-04-25T12:06:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.355+0000 7fdaba89a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fdab410ddb0 con 0x7fdab4105360 2024-04-25T12:06:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.355+0000 7fdab8e37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab4107c90 0x7fdab419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.355+0000 7fdab8e37640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab4107c90 0x7fdab419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40996/0 (socket says 172.21.15.19:40996) 2024-04-25T12:06:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.355+0000 7fdab8e37640 1 -- 172.21.15.19:0/1332553172 learned_addr learned my addr 172.21.15.19:0/1332553172 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.356+0000 7fdab37fe640 1 --2- 172.21.15.19:0/1332553172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab4105360 0x7fdab41987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.356+0000 7fdab3fff640 1 --2- 172.21.15.19:0/1332553172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102a30 0x7fdab4198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.356+0000 7fdab8e37640 1 -- 172.21.15.19:0/1332553172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab4105360 msgr2=0x7fdab41987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.356+0000 7fdab8e37640 1 --2- 172.21.15.19:0/1332553172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab4105360 0x7fdab41987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.356+0000 7fdab8e37640 1 -- 172.21.15.19:0/1332553172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102a30 msgr2=0x7fdab4198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.356+0000 7fdab8e37640 1 --2- 172.21.15.19:0/1332553172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102a30 0x7fdab4198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.356+0000 7fdab8e37640 1 -- 172.21.15.19:0/1332553172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab419fe40 con 0x7fdab4107c90 2024-04-25T12:06:31.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.356+0000 7fdab8e37640 1 --2- 172.21.15.19:0/1332553172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab4107c90 0x7fdab419f7d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009dc0 tx=0x7fdaac031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:31.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.356+0000 7fdab17fa640 1 -- 172.21.15.19:0/1332553172 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac003bf0 con 0x7fdab4107c90 2024-04-25T12:06:31.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.356+0000 7fdab17fa640 1 -- 172.21.15.19:0/1332553172 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fdaac003d90 con 0x7fdab4107c90 2024-04-25T12:06:31.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.356+0000 7fdaba89a640 1 -- 172.21.15.19:0/1332553172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab41a00d0 con 0x7fdab4107c90 2024-04-25T12:06:31.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.356+0000 7fdab17fa640 1 -- 172.21.15.19:0/1332553172 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac02fcd0 con 0x7fdab4107c90 2024-04-25T12:06:31.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.356+0000 7fdaba89a640 1 -- 172.21.15.19:0/1332553172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab41a0620 con 0x7fdab4107c90 2024-04-25T12:06:31.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.357+0000 7fdaba89a640 1 -- 172.21.15.19:0/1332553172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab40fdf10 con 0x7fdab4107c90 2024-04-25T12:06:31.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.358+0000 7fdab17fa640 1 -- 172.21.15.19:0/1332553172 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdaac004040 con 0x7fdab4107c90 2024-04-25T12:06:31.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.359+0000 7fdab17fa640 1 --2- 172.21.15.19:0/1332553172 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fda9c0a3bb0 0x7fda9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.359+0000 7fdab3fff640 1 --2- 172.21.15.19:0/1332553172 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fda9c0a3bb0 0x7fda9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.359+0000 7fdab17fa640 1 -- 172.21.15.19:0/1332553172 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fdaac0f1b30 con 0x7fdab4107c90 2024-04-25T12:06:31.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.360+0000 7fdab3fff640 1 --2- 172.21.15.19:0/1332553172 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fda9c0a3bb0 0x7fda9c0a6070 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009aa0 tx=0x7fdaa8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:31.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.365+0000 7fdab17fa640 1 -- 172.21.15.19:0/1332553172 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdaac0be490 con 0x7fdab4107c90 2024-04-25T12:06:31.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.550+0000 7fdaba89a640 1 -- 172.21.15.19:0/1332553172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdab4062b50 con 0x7fdab4107c90 2024-04-25T12:06:31.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.550+0000 7fdab17fa640 1 -- 172.21.15.19:0/1332553172 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fdaac0bdd90 con 0x7fdab4107c90 2024-04-25T12:06:31.550 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T12:06:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.553+0000 7fdaba89a640 1 -- 172.21.15.19:0/1332553172 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fda9c0a3bb0 msgr2=0x7fda9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.553+0000 7fdaba89a640 1 --2- 172.21.15.19:0/1332553172 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fda9c0a3bb0 0x7fda9c0a6070 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009aa0 tx=0x7fdaa8005f90 comp rx=0 tx=0).stop 2024-04-25T12:06:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.553+0000 7fdaba89a640 1 -- 172.21.15.19:0/1332553172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab4107c90 msgr2=0x7fdab419f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.553+0000 7fdaba89a640 1 --2- 172.21.15.19:0/1332553172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab4107c90 0x7fdab419f7d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009dc0 tx=0x7fdaac031840 comp rx=0 tx=0).stop 2024-04-25T12:06:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.553+0000 7fdaba89a640 1 -- 172.21.15.19:0/1332553172 shutdown_connections 2024-04-25T12:06:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.553+0000 7fdaba89a640 1 --2- 172.21.15.19:0/1332553172 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fda9c0a3bb0 0x7fda9c0a6070 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.553+0000 7fdaba89a640 1 --2- 172.21.15.19:0/1332553172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdab4107c90 0x7fdab419f7d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.553+0000 7fdaba89a640 1 --2- 172.21.15.19:0/1332553172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdab4105360 0x7fdab41987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.553+0000 7fdaba89a640 1 --2- 172.21.15.19:0/1332553172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdab4102a30 0x7fdab4198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.553+0000 7fdaba89a640 1 -- 172.21.15.19:0/1332553172 >> 172.21.15.19:0/1332553172 conn(0x7fdab40fc420 msgr2=0x7fdab4103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.553+0000 7fdaba89a640 1 -- 172.21.15.19:0/1332553172 shutdown_connections 2024-04-25T12:06:31.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.553+0000 7fdaba89a640 1 -- 172.21.15.19:0/1332553172 wait complete. 2024-04-25T12:06:31.563 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T12:06:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.670+0000 7f9ed8a59640 1 -- 172.21.15.19:0/739038625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4107c90 msgr2=0x7f9ed410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.670+0000 7f9ed8a59640 1 --2- 172.21.15.19:0/739038625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4107c90 0x7f9ed410a0e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009ca0 tx=0x7f9ecc02a8c0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.670+0000 7f9ed8a59640 1 -- 172.21.15.19:0/739038625 shutdown_connections 2024-04-25T12:06:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.670+0000 7f9ed8a59640 1 --2- 172.21.15.19:0/739038625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4107c90 0x7f9ed410a0e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.670+0000 7f9ed8a59640 1 --2- 172.21.15.19:0/739038625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ed4105360 0x7f9ed4107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.670+0000 7f9ed8a59640 1 --2- 172.21.15.19:0/739038625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed4102a30 0x7f9ed4104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.670+0000 7f9ed8a59640 1 -- 172.21.15.19:0/739038625 >> 172.21.15.19:0/739038625 conn(0x7f9ed40fc440 msgr2=0x7f9ed40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.670+0000 7f9ed8a59640 1 -- 172.21.15.19:0/739038625 shutdown_connections 2024-04-25T12:06:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.670+0000 7f9ed8a59640 1 -- 172.21.15.19:0/739038625 wait complete. 2024-04-25T12:06:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.671+0000 7f9ed8a59640 1 Processor -- start 2024-04-25T12:06:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.671+0000 7f9ed8a59640 1 -- start start 2024-04-25T12:06:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.671+0000 7f9ed8a59640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed4102a30 0x7f9ed41982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.671+0000 7f9ed8a59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4105360 0x7f9ed4198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.671+0000 7f9ed8a59640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ed4107c90 0x7f9ed419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.671+0000 7f9ed8a59640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed410dc80 con 0x7f9ed4105360 2024-04-25T12:06:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.671+0000 7f9ed8a59640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed410db00 con 0x7f9ed4107c90 2024-04-25T12:06:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.671+0000 7f9ed8a59640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed410de00 con 0x7f9ed4102a30 2024-04-25T12:06:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.671+0000 7f9ed1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4105360 0x7f9ed4198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.671+0000 7f9ed1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4105360 0x7f9ed4198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36216/0 (socket says 172.21.15.19:36216) 2024-04-25T12:06:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.671+0000 7f9ed259c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed4102a30 0x7f9ed41982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.671+0000 7f9ed1d9b640 1 -- 172.21.15.19:0/1673372588 learned_addr learned my addr 172.21.15.19:0/1673372588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.671+0000 7f9ed2d9d640 1 --2- 172.21.15.19:0/1673372588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ed4107c90 0x7f9ed419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.672+0000 7f9ed259c640 1 -- 172.21.15.19:0/1673372588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ed4107c90 msgr2=0x7f9ed419f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.672+0000 7f9ed259c640 1 --2- 172.21.15.19:0/1673372588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ed4107c90 0x7f9ed419f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.672+0000 7f9ed259c640 1 -- 172.21.15.19:0/1673372588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4105360 msgr2=0x7f9ed4198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.672+0000 7f9ed259c640 1 --2- 172.21.15.19:0/1673372588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4105360 0x7f9ed4198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.672+0000 7f9ed259c640 1 -- 172.21.15.19:0/1673372588 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed419fef0 con 0x7f9ed4102a30 2024-04-25T12:06:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.672+0000 7f9ed2d9d640 1 --2- 172.21.15.19:0/1673372588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ed4107c90 0x7f9ed419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:31.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.672+0000 7f9ed259c640 1 --2- 172.21.15.19:0/1673372588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed4102a30 0x7f9ed41982d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400e9f0 tx=0x7f9ec400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:31.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.673+0000 7f9ec37fe640 1 -- 172.21.15.19:0/1673372588 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec400ccc0 con 0x7f9ed4102a30 2024-04-25T12:06:31.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.673+0000 7f9ec37fe640 1 -- 172.21.15.19:0/1673372588 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9ec4005020 con 0x7f9ed4102a30 2024-04-25T12:06:31.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.673+0000 7f9ec37fe640 1 -- 172.21.15.19:0/1673372588 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec40049c0 con 0x7f9ed4102a30 2024-04-25T12:06:31.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.673+0000 7f9ed8a59640 1 -- 172.21.15.19:0/1673372588 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed41a01e0 con 0x7f9ed4102a30 2024-04-25T12:06:31.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.673+0000 7f9ed8a59640 1 -- 172.21.15.19:0/1673372588 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed41a0730 con 0x7f9ed4102a30 2024-04-25T12:06:31.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.674+0000 7f9ed8a59640 1 -- 172.21.15.19:0/1673372588 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed40fdf30 con 0x7f9ed4102a30 2024-04-25T12:06:31.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.675+0000 7f9ec37fe640 1 -- 172.21.15.19:0/1673372588 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9ec40051d0 con 0x7f9ed4102a30 2024-04-25T12:06:31.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.676+0000 7f9ec37fe640 1 --2- 172.21.15.19:0/1673372588 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9ebc0a3c30 0x7f9ebc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.676+0000 7f9ed1d9b640 1 --2- 172.21.15.19:0/1673372588 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9ebc0a3c30 0x7f9ebc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.676+0000 7f9ec37fe640 1 -- 172.21.15.19:0/1673372588 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9ec40d05f0 con 0x7f9ed4102a30 2024-04-25T12:06:31.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.676+0000 7f9ed1d9b640 1 --2- 172.21.15.19:0/1673372588 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9ebc0a3c30 0x7f9ebc0a60f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9ed41997e0 tx=0x7f9ec8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:31.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.680+0000 7f9ec37fe640 1 -- 172.21.15.19:0/1673372588 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ec409cf50 con 0x7f9ed4102a30 2024-04-25T12:06:31.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.869+0000 7f9ed8a59640 1 -- 172.21.15.19:0/1673372588 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9ed4062b50 con 0x7f9ed4102a30 2024-04-25T12:06:31.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.869+0000 7f9ec37fe640 1 -- 172.21.15.19:0/1673372588 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f9ec409c850 con 0x7f9ed4102a30 2024-04-25T12:06:31.869 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T12:06:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.871+0000 7f9ed8a59640 1 -- 172.21.15.19:0/1673372588 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9ebc0a3c30 msgr2=0x7f9ebc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.871+0000 7f9ed8a59640 1 --2- 172.21.15.19:0/1673372588 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9ebc0a3c30 0x7f9ebc0a60f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9ed41997e0 tx=0x7f9ec8005f90 comp rx=0 tx=0).stop 2024-04-25T12:06:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.871+0000 7f9ed8a59640 1 -- 172.21.15.19:0/1673372588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed4102a30 msgr2=0x7f9ed41982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.871+0000 7f9ed8a59640 1 --2- 172.21.15.19:0/1673372588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed4102a30 0x7f9ed41982d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400e9f0 tx=0x7f9ec400eeb0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.872+0000 7f9ed8a59640 1 -- 172.21.15.19:0/1673372588 shutdown_connections 2024-04-25T12:06:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.872+0000 7f9ed8a59640 1 --2- 172.21.15.19:0/1673372588 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9ebc0a3c30 0x7f9ebc0a60f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.872+0000 7f9ed8a59640 1 --2- 172.21.15.19:0/1673372588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ed4107c90 0x7f9ed419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.872+0000 7f9ed8a59640 1 --2- 172.21.15.19:0/1673372588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ed4105360 0x7f9ed4198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.872+0000 7f9ed8a59640 1 --2- 172.21.15.19:0/1673372588 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed4102a30 0x7f9ed41982d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.872+0000 7f9ed8a59640 1 -- 172.21.15.19:0/1673372588 >> 172.21.15.19:0/1673372588 conn(0x7f9ed40fc440 msgr2=0x7f9ed4103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.872+0000 7f9ed8a59640 1 -- 172.21.15.19:0/1673372588 shutdown_connections 2024-04-25T12:06:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.872+0000 7f9ed8a59640 1 -- 172.21.15.19:0/1673372588 wait complete. 2024-04-25T12:06:31.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:31 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/51563641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:06:31.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:31 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/683085108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:06:31.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:31 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1332553172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:06:31.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:06:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.985+0000 7f6b3a210640 1 -- 172.21.15.19:0/2457155876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34105360 msgr2=0x7f6b34107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.985+0000 7f6b3a210640 1 --2- 172.21.15.19:0/2457155876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34105360 0x7f6b34107750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009900 tx=0x7f6b2802a410 comp rx=0 tx=0).stop 2024-04-25T12:06:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.985+0000 7f6b3a210640 1 -- 172.21.15.19:0/2457155876 shutdown_connections 2024-04-25T12:06:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.985+0000 7f6b3a210640 1 --2- 172.21.15.19:0/2457155876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b34107c90 0x7f6b3410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.985+0000 7f6b3a210640 1 --2- 172.21.15.19:0/2457155876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34105360 0x7f6b34107750 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.985+0000 7f6b3a210640 1 --2- 172.21.15.19:0/2457155876 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b34102a30 0x7f6b34104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.985+0000 7f6b3a210640 1 -- 172.21.15.19:0/2457155876 >> 172.21.15.19:0/2457155876 conn(0x7f6b340fc440 msgr2=0x7f6b340fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.985+0000 7f6b3a210640 1 -- 172.21.15.19:0/2457155876 shutdown_connections 2024-04-25T12:06:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.985+0000 7f6b3a210640 1 -- 172.21.15.19:0/2457155876 wait complete. 2024-04-25T12:06:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.985+0000 7f6b3a210640 1 Processor -- start 2024-04-25T12:06:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b3a210640 1 -- start start 2024-04-25T12:06:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b3a210640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b34102a30 0x7f6b34198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b3a210640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b34105360 0x7f6b341987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b3a210640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34107c90 0x7f6b3419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b3a210640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3410dc10 con 0x7f6b34107c90 2024-04-25T12:06:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b3a210640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3410da90 con 0x7f6b34105360 2024-04-25T12:06:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b3a210640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3410dd90 con 0x7f6b34102a30 2024-04-25T12:06:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b33fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34107c90 0x7f6b3419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b33fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34107c90 0x7f6b3419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36226/0 (socket says 172.21.15.19:36226) 2024-04-25T12:06:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b33fff640 1 -- 172.21.15.19:0/3843833814 learned_addr learned my addr 172.21.15.19:0/3843833814 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b32ffd640 1 --2- 172.21.15.19:0/3843833814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b34105360 0x7f6b341987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b337fe640 1 --2- 172.21.15.19:0/3843833814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b34102a30 0x7f6b34198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b33fff640 1 -- 172.21.15.19:0/3843833814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b34102a30 msgr2=0x7f6b34198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b33fff640 1 --2- 172.21.15.19:0/3843833814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b34102a30 0x7f6b34198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b33fff640 1 -- 172.21.15.19:0/3843833814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b34105360 msgr2=0x7f6b341987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b33fff640 1 --2- 172.21.15.19:0/3843833814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b34105360 0x7f6b341987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b33fff640 1 -- 172.21.15.19:0/3843833814 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3419fe80 con 0x7f6b34107c90 2024-04-25T12:06:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b33fff640 1 --2- 172.21.15.19:0/3843833814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34107c90 0x7f6b3419f810 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00eaa0 tx=0x7f6b2c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.986+0000 7f6b30ff9640 1 -- 172.21.15.19:0/3843833814 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c00cdb0 con 0x7f6b34107c90 2024-04-25T12:06:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.987+0000 7f6b3a210640 1 -- 172.21.15.19:0/3843833814 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b341a0170 con 0x7f6b34107c90 2024-04-25T12:06:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.987+0000 7f6b30ff9640 1 -- 172.21.15.19:0/3843833814 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6b2c005020 con 0x7f6b34107c90 2024-04-25T12:06:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.987+0000 7f6b30ff9640 1 -- 172.21.15.19:0/3843833814 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0116d0 con 0x7f6b34107c90 2024-04-25T12:06:31.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.987+0000 7f6b3a210640 1 -- 172.21.15.19:0/3843833814 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b341a0690 con 0x7f6b34107c90 2024-04-25T12:06:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.988+0000 7f6b3a210640 1 -- 172.21.15.19:0/3843833814 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b340fdf30 con 0x7f6b34107c90 2024-04-25T12:06:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.989+0000 7f6b30ff9640 1 -- 172.21.15.19:0/3843833814 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6b2c0051d0 con 0x7f6b34107c90 2024-04-25T12:06:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.989+0000 7f6b30ff9640 1 --2- 172.21.15.19:0/3843833814 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6b1c0a3bb0 0x7f6b1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.989+0000 7f6b30ff9640 1 -- 172.21.15.19:0/3843833814 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f6b2c0d0710 con 0x7f6b34107c90 2024-04-25T12:06:31.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.989+0000 7f6b337fe640 1 --2- 172.21.15.19:0/3843833814 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6b1c0a3bb0 0x7f6b1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:31.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.990+0000 7f6b337fe640 1 --2- 172.21.15.19:0/3843833814 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6b1c0a3bb0 0x7f6b1c0a6070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6b24005e10 tx=0x7f6b24005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:31.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:31.995+0000 7f6b30ff9640 1 -- 172.21.15.19:0/3843833814 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b2c016030 con 0x7f6b34107c90 2024-04-25T12:06:32.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:31 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/51563641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:06:32.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:31 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/683085108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:06:32.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:31 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1332553172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:06:32.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:31 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/51563641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:06:32.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:31 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/683085108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:06:32.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:31 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1332553172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:06:32.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.220+0000 7f6b3a210640 1 -- 172.21.15.19:0/3843833814 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b34062b50 con 0x7f6b34107c90 2024-04-25T12:06:32.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.221+0000 7f6b30ff9640 1 -- 172.21.15.19:0/3843833814 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11944 (secure 0 0 0) 0x7f6b2c09d0d0 con 0x7f6b34107c90 2024-04-25T12:06:32.220 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:32.220 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T12:05:04.008679+0000","modified":"2024-04-25T12:05:11.186222+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24514,"mds_1":24500,"mds_2":24496,"mds_3":24484,"mds_4":14655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/1694897630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1694897630},{"type":"v1","addr":"172.21.15.19:6839","nonce":1694897630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6841/3247157172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3247157172},{"type":"v1","addr":"172.21.15.19:6841","nonce":3247157172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/3918801450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3918801450},{"type":"v1","addr":"172.21.15.150:6837","nonce":3918801450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2553528303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2553528303},{"type":"v1","addr":"172.21.15.150:6839","nonce":2553528303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/1723439500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1723439500},{"type":"v1","addr":"172.21.15.169:6839","nonce":1723439500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24500,24514,24496,24484,14655]},"id":1}]} 2024-04-25T12:06:32.221 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-25T12:06:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.224+0000 7f6b3a210640 1 -- 172.21.15.19:0/3843833814 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6b1c0a3bb0 msgr2=0x7f6b1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.224+0000 7f6b3a210640 1 --2- 172.21.15.19:0/3843833814 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6b1c0a3bb0 0x7f6b1c0a6070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6b24005e10 tx=0x7f6b24005da0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.224+0000 7f6b3a210640 1 -- 172.21.15.19:0/3843833814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34107c90 msgr2=0x7f6b3419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.224+0000 7f6b3a210640 1 --2- 172.21.15.19:0/3843833814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34107c90 0x7f6b3419f810 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00eaa0 tx=0x7f6b2c00ef60 comp rx=0 tx=0).stop 2024-04-25T12:06:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.224+0000 7f6b3a210640 1 -- 172.21.15.19:0/3843833814 shutdown_connections 2024-04-25T12:06:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.224+0000 7f6b3a210640 1 --2- 172.21.15.19:0/3843833814 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6b1c0a3bb0 0x7f6b1c0a6070 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.224+0000 7f6b3a210640 1 --2- 172.21.15.19:0/3843833814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b34107c90 0x7f6b3419f810 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.224+0000 7f6b3a210640 1 --2- 172.21.15.19:0/3843833814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b34105360 0x7f6b341987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.224+0000 7f6b3a210640 1 --2- 172.21.15.19:0/3843833814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b34102a30 0x7f6b34198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.224+0000 7f6b3a210640 1 -- 172.21.15.19:0/3843833814 >> 172.21.15.19:0/3843833814 conn(0x7f6b340fc440 msgr2=0x7f6b341032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.224+0000 7f6b3a210640 1 -- 172.21.15.19:0/3843833814 shutdown_connections 2024-04-25T12:06:32.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.224+0000 7f6b3a210640 1 -- 172.21.15.19:0/3843833814 wait complete. 2024-04-25T12:06:32.234 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:06:32.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.337+0000 7fcb03ee1640 1 -- 172.21.15.19:0/3670834899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcafc101fa0 msgr2=0x7fcafc1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.337+0000 7fcb03ee1640 1 --2- 172.21.15.19:0/3670834899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcafc101fa0 0x7fcafc1023c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009900 tx=0x7fcaf402a410 comp rx=0 tx=0).stop 2024-04-25T12:06:32.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.337+0000 7fcb03ee1640 1 -- 172.21.15.19:0/3670834899 shutdown_connections 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.337+0000 7fcb03ee1640 1 --2- 172.21.15.19:0/3670834899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcafc103b50 0x7fcafc10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.337+0000 7fcb03ee1640 1 --2- 172.21.15.19:0/3670834899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcafc103190 0x7fcafc103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.337+0000 7fcb03ee1640 1 --2- 172.21.15.19:0/3670834899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcafc101fa0 0x7fcafc1023c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.337+0000 7fcb03ee1640 1 -- 172.21.15.19:0/3670834899 >> 172.21.15.19:0/3670834899 conn(0x7fcafc0fd550 msgr2=0x7fcafc0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.337+0000 7fcb03ee1640 1 -- 172.21.15.19:0/3670834899 shutdown_connections 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.337+0000 7fcb03ee1640 1 -- 172.21.15.19:0/3670834899 wait complete. 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb03ee1640 1 Processor -- start 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb03ee1640 1 -- start start 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb03ee1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcafc101fa0 0x7fcafc198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb03ee1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcafc103190 0x7fcafc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb03ee1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcafc103b50 0x7fcafc19fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb03ee1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc10dce0 con 0x7fcafc103b50 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb03ee1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc10db60 con 0x7fcafc103190 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb03ee1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc10de60 con 0x7fcafc101fa0 2024-04-25T12:06:32.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb0247e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcafc103b50 0x7fcafc19fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb0247e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcafc103b50 0x7fcafc19fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36246/0 (socket says 172.21.15.19:36246) 2024-04-25T12:06:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb0247e640 1 -- 172.21.15.19:0/1060818490 learned_addr learned my addr 172.21.15.19:0/1060818490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb01c7d640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcafc101fa0 0x7fcafc198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb0147c640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcafc103190 0x7fcafc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb0247e640 1 -- 172.21.15.19:0/1060818490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcafc101fa0 msgr2=0x7fcafc198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb0247e640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcafc101fa0 0x7fcafc198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb0247e640 1 -- 172.21.15.19:0/1060818490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcafc103190 msgr2=0x7fcafc198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb0247e640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcafc103190 0x7fcafc198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb0247e640 1 -- 172.21.15.19:0/1060818490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc1a0070 con 0x7fcafc103b50 2024-04-25T12:06:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb01c7d640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcafc101fa0 0x7fcafc198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb0247e640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcafc103b50 0x7fcafc19fa00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009fd0 tx=0x7fcaec00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcafaffd640 1 -- 172.21.15.19:0/1060818490 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec011830 con 0x7fcafc103b50 2024-04-25T12:06:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcafaffd640 1 -- 172.21.15.19:0/1060818490 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fcaec016070 con 0x7fcafc103b50 2024-04-25T12:06:32.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcb03ee1640 1 -- 172.21.15.19:0/1060818490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcafc1a0360 con 0x7fcafc103b50 2024-04-25T12:06:32.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.338+0000 7fcafaffd640 1 -- 172.21.15.19:0/1060818490 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec00f2e0 con 0x7fcafc103b50 2024-04-25T12:06:32.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.339+0000 7fcb03ee1640 1 -- 172.21.15.19:0/1060818490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcafc1a08e0 con 0x7fcafc103b50 2024-04-25T12:06:32.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.340+0000 7fcb03ee1640 1 -- 172.21.15.19:0/1060818490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcafc109330 con 0x7fcafc103b50 2024-04-25T12:06:32.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.341+0000 7fcafaffd640 1 -- 172.21.15.19:0/1060818490 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcaec0119d0 con 0x7fcafc103b50 2024-04-25T12:06:32.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.341+0000 7fcafaffd640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fcae40a3bb0 0x7fcae40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.341+0000 7fcb01c7d640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fcae40a3bb0 0x7fcae40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.341+0000 7fcafaffd640 1 -- 172.21.15.19:0/1060818490 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fcaec0d0250 con 0x7fcafc103b50 2024-04-25T12:06:32.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.342+0000 7fcb01c7d640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fcae40a3bb0 0x7fcae40a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4000c00 tx=0x7fcaf4031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:32.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.346+0000 7fcafaffd640 1 -- 172.21.15.19:0/1060818490 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcaec09cc10 con 0x7fcafc103b50 2024-04-25T12:06:32.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.534+0000 7fcb03ee1640 1 -- 172.21.15.19:0/1060818490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcafc062b50 con 0x7fcafc103b50 2024-04-25T12:06:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.535+0000 7fcafaffd640 1 -- 172.21.15.19:0/1060818490 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22731 (secure 0 0 0) 0x7fcaec09c510 con 0x7fcafc103b50 2024-04-25T12:06:32.535 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:32.535 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":87,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:05:55.975412+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:05:01.658354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:05:02.969078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:06:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.537+0000 7fcb03ee1640 1 -- 172.21.15.19:0/1060818490 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fcae40a3bb0 msgr2=0x7fcae40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.537+0000 7fcb03ee1640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fcae40a3bb0 0x7fcae40a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4000c00 tx=0x7fcaf4031510 comp rx=0 tx=0).stop 2024-04-25T12:06:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.537+0000 7fcb03ee1640 1 -- 172.21.15.19:0/1060818490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcafc103b50 msgr2=0x7fcafc19fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.537+0000 7fcb03ee1640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcafc103b50 0x7fcafc19fa00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009fd0 tx=0x7fcaec00b8e0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.537+0000 7fcb03ee1640 1 -- 172.21.15.19:0/1060818490 shutdown_connections 2024-04-25T12:06:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.537+0000 7fcb03ee1640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fcae40a3bb0 0x7fcae40a6070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.537+0000 7fcb03ee1640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcafc103b50 0x7fcafc19fa00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.537+0000 7fcb03ee1640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcafc103190 0x7fcafc198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.537+0000 7fcb03ee1640 1 --2- 172.21.15.19:0/1060818490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcafc101fa0 0x7fcafc198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.537+0000 7fcb03ee1640 1 -- 172.21.15.19:0/1060818490 >> 172.21.15.19:0/1060818490 conn(0x7fcafc0fd550 msgr2=0x7fcafc106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.537+0000 7fcb03ee1640 1 -- 172.21.15.19:0/1060818490 shutdown_connections 2024-04-25T12:06:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.537+0000 7fcb03ee1640 1 -- 172.21.15.19:0/1060818490 wait complete. 2024-04-25T12:06:32.548 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:00:20.898809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:05:01.658354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:05:02.969078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T12:06:32.549 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:06:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 -- 172.21.15.19:0/1543884124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7674100e90 msgr2=0x7f76741012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 --2- 172.21.15.19:0/1543884124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7674100e90 0x7f76741012b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7670009900 tx=0x7f767002a410 comp rx=0 tx=0).stop 2024-04-25T12:06:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 -- 172.21.15.19:0/1543884124 shutdown_connections 2024-04-25T12:06:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 --2- 172.21.15.19:0/1543884124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76741029a0 0x7f76741092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 --2- 172.21.15.19:0/1543884124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7674101fe0 0x7f7674102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 --2- 172.21.15.19:0/1543884124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7674100e90 0x7f76741012b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 -- 172.21.15.19:0/1543884124 >> 172.21.15.19:0/1543884124 conn(0x7f76740fc400 msgr2=0x7f76740fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 -- 172.21.15.19:0/1543884124 shutdown_connections 2024-04-25T12:06:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 -- 172.21.15.19:0/1543884124 wait complete. 2024-04-25T12:06:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 Processor -- start 2024-04-25T12:06:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 -- start start 2024-04-25T12:06:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7674100e90 0x7f7674069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7674101fe0 0x7f767419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76741029a0 0x7f767419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f767410cb00 con 0x7f7674101fe0 2024-04-25T12:06:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f767410c980 con 0x7f7674100e90 2024-04-25T12:06:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f767baea640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f767410cc80 con 0x7f76741029a0 2024-04-25T12:06:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f7679085640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7674101fe0 0x7f767419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f7679085640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7674101fe0 0x7f767419cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36260/0 (socket says 172.21.15.19:36260) 2024-04-25T12:06:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.655+0000 7f7679085640 1 -- 172.21.15.19:0/887307391 learned_addr learned my addr 172.21.15.19:0/887307391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.657+0000 7f7679886640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7674100e90 0x7f7674069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.657+0000 7f767a087640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76741029a0 0x7f767419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.657+0000 7f7679085640 1 -- 172.21.15.19:0/887307391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76741029a0 msgr2=0x7f767419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.657+0000 7f7679085640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76741029a0 0x7f767419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.657+0000 7f7679085640 1 -- 172.21.15.19:0/887307391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7674100e90 msgr2=0x7f7674069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.657+0000 7f7679085640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7674100e90 0x7f7674069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.657+0000 7f7679085640 1 -- 172.21.15.19:0/887307391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767419fee0 con 0x7f7674101fe0 2024-04-25T12:06:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.657+0000 7f7679886640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7674100e90 0x7f7674069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:06:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.657+0000 7f7679085640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7674101fe0 0x7f767419cfe0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f766400c970 tx=0x7f766400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.657+0000 7f766affd640 1 -- 172.21.15.19:0/887307391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664007c10 con 0x7f7674101fe0 2024-04-25T12:06:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.657+0000 7f766affd640 1 -- 172.21.15.19:0/887307391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f7664007db0 con 0x7f7674101fe0 2024-04-25T12:06:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.657+0000 7f767baea640 1 -- 172.21.15.19:0/887307391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76741a01d0 con 0x7f7674101fe0 2024-04-25T12:06:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.657+0000 7f766affd640 1 -- 172.21.15.19:0/887307391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766400f910 con 0x7f7674101fe0 2024-04-25T12:06:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.657+0000 7f767baea640 1 -- 172.21.15.19:0/887307391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76741a0720 con 0x7f7674101fe0 2024-04-25T12:06:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.658+0000 7f767baea640 1 -- 172.21.15.19:0/887307391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76741081a0 con 0x7f7674101fe0 2024-04-25T12:06:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.659+0000 7f766affd640 1 -- 172.21.15.19:0/887307391 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f766400fab0 con 0x7f7674101fe0 2024-04-25T12:06:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.660+0000 7f766affd640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f76600a3bb0 0x7f76600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.660+0000 7f7679886640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f76600a3bb0 0x7f76600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.660+0000 7f766affd640 1 -- 172.21.15.19:0/887307391 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f76640d08d0 con 0x7f7674101fe0 2024-04-25T12:06:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.660+0000 7f7679886640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f76600a3bb0 0x7f76600a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f76700096c0 tx=0x7f7670009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:32.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.664+0000 7f766affd640 1 -- 172.21.15.19:0/887307391 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f766409d2c0 con 0x7f7674101fe0 2024-04-25T12:06:32.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:32 smithi019 ceph-mon[29213]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T12:06:32.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:32 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1673372588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:06:32.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:32 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3843833814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:06:32.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:32 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1060818490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:06:32.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.854+0000 7f767baea640 1 -- 172.21.15.19:0/887307391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7674062b50 con 0x7f7674101fe0 2024-04-25T12:06:32.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.855+0000 7f766affd640 1 -- 172.21.15.19:0/887307391 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f766409cbc0 con 0x7f7674101fe0 2024-04-25T12:06:32.853 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T12:06:32.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.857+0000 7f767baea640 1 -- 172.21.15.19:0/887307391 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f76600a3bb0 msgr2=0x7f76600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.857+0000 7f767baea640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f76600a3bb0 0x7f76600a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f76700096c0 tx=0x7f7670009650 comp rx=0 tx=0).stop 2024-04-25T12:06:32.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.857+0000 7f767baea640 1 -- 172.21.15.19:0/887307391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7674101fe0 msgr2=0x7f767419cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.857+0000 7f767baea640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7674101fe0 0x7f767419cfe0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f766400c970 tx=0x7f766400ce30 comp rx=0 tx=0).stop 2024-04-25T12:06:32.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.857+0000 7f767baea640 1 -- 172.21.15.19:0/887307391 shutdown_connections 2024-04-25T12:06:32.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.857+0000 7f767baea640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f76600a3bb0 0x7f76600a6070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.857+0000 7f767baea640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76741029a0 0x7f767419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.857+0000 7f767baea640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7674101fe0 0x7f767419cfe0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.857+0000 7f767baea640 1 --2- 172.21.15.19:0/887307391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7674100e90 0x7f7674069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.857+0000 7f767baea640 1 -- 172.21.15.19:0/887307391 >> 172.21.15.19:0/887307391 conn(0x7f76740fc400 msgr2=0x7f7674105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:32.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.857+0000 7f767baea640 1 -- 172.21.15.19:0/887307391 shutdown_connections 2024-04-25T12:06:32.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.857+0000 7f767baea640 1 -- 172.21.15.19:0/887307391 wait complete. 2024-04-25T12:06:32.866 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T12:06:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.972+0000 7f4f78e31640 1 -- 172.21.15.19:0/468458925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f74100e70 msgr2=0x7f4f74101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.972+0000 7f4f78e31640 1 --2- 172.21.15.19:0/468458925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f74100e70 0x7f4f74101290 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009900 tx=0x7f4f6402a410 comp rx=0 tx=0).stop 2024-04-25T12:06:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.972+0000 7f4f78e31640 1 -- 172.21.15.19:0/468458925 shutdown_connections 2024-04-25T12:06:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.972+0000 7f4f78e31640 1 --2- 172.21.15.19:0/468458925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f74102980 0x7f4f741092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.972+0000 7f4f78e31640 1 --2- 172.21.15.19:0/468458925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f74101fc0 0x7f4f74102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.972+0000 7f4f78e31640 1 --2- 172.21.15.19:0/468458925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f74100e70 0x7f4f74101290 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.972+0000 7f4f78e31640 1 -- 172.21.15.19:0/468458925 >> 172.21.15.19:0/468458925 conn(0x7f4f740fc3e0 msgr2=0x7f4f740fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.972+0000 7f4f78e31640 1 -- 172.21.15.19:0/468458925 shutdown_connections 2024-04-25T12:06:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.972+0000 7f4f78e31640 1 -- 172.21.15.19:0/468458925 wait complete. 2024-04-25T12:06:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.973+0000 7f4f78e31640 1 Processor -- start 2024-04-25T12:06:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.973+0000 7f4f78e31640 1 -- start start 2024-04-25T12:06:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.973+0000 7f4f78e31640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f74100e70 0x7f4f74069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.973+0000 7f4f78e31640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f74101fc0 0x7f4f7419d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.973+0000 7f4f78e31640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f74102980 0x7f4f7419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.973+0000 7f4f78e31640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7410cc50 con 0x7f4f74100e70 2024-04-25T12:06:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.973+0000 7f4f78e31640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7410cad0 con 0x7f4f74101fc0 2024-04-25T12:06:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.973+0000 7f4f78e31640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7410cdd0 con 0x7f4f74102980 2024-04-25T12:06:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.973+0000 7f4f7259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f74100e70 0x7f4f74069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.973+0000 7f4f7259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f74100e70 0x7f4f74069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36286/0 (socket says 172.21.15.19:36286) 2024-04-25T12:06:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.973+0000 7f4f7259c640 1 -- 172.21.15.19:0/3110744457 learned_addr learned my addr 172.21.15.19:0/3110744457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.974+0000 7f4f72d9d640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f74102980 0x7f4f7419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.974+0000 7f4f71d9b640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f74101fc0 0x7f4f7419d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.974+0000 7f4f7259c640 1 -- 172.21.15.19:0/3110744457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f74102980 msgr2=0x7f4f7419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.974+0000 7f4f7259c640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f74102980 0x7f4f7419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.974+0000 7f4f7259c640 1 -- 172.21.15.19:0/3110744457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f74101fc0 msgr2=0x7f4f7419d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.974+0000 7f4f7259c640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f74101fc0 0x7f4f7419d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.974+0000 7f4f7259c640 1 -- 172.21.15.19:0/3110744457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f741a00d0 con 0x7f4f74100e70 2024-04-25T12:06:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.974+0000 7f4f72d9d640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f74102980 0x7f4f7419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.974+0000 7f4f7259c640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f74100e70 0x7f4f74069c50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4f64031570 tx=0x7f4f64032d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.974+0000 7f4f637fe640 1 -- 172.21.15.19:0/3110744457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6403f030 con 0x7f4f74100e70 2024-04-25T12:06:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.974+0000 7f4f637fe640 1 -- 172.21.15.19:0/3110744457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4f64003d90 con 0x7f4f74100e70 2024-04-25T12:06:32.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.974+0000 7f4f78e31640 1 -- 172.21.15.19:0/3110744457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f741a0300 con 0x7f4f74100e70 2024-04-25T12:06:32.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.974+0000 7f4f637fe640 1 -- 172.21.15.19:0/3110744457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64035070 con 0x7f4f74100e70 2024-04-25T12:06:32.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.974+0000 7f4f78e31640 1 -- 172.21.15.19:0/3110744457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f741a0870 con 0x7f4f74100e70 2024-04-25T12:06:32.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.975+0000 7f4f78e31640 1 -- 172.21.15.19:0/3110744457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f741081f0 con 0x7f4f74100e70 2024-04-25T12:06:32.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.976+0000 7f4f637fe640 1 -- 172.21.15.19:0/3110744457 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4f64042030 con 0x7f4f74100e70 2024-04-25T12:06:32.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.977+0000 7f4f637fe640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4f5c0a3bb0 0x7f4f5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:32.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.977+0000 7f4f71d9b640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4f5c0a3bb0 0x7f4f5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:32.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.977+0000 7f4f637fe640 1 -- 172.21.15.19:0/3110744457 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f4f640f0b90 con 0x7f4f74100e70 2024-04-25T12:06:32.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.978+0000 7f4f71d9b640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4f5c0a3bb0 0x7f4f5c0a6070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4f740699c0 tx=0x7f4f68008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:32.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:32.982+0000 7f4f637fe640 1 -- 172.21.15.19:0/3110744457 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f640bd550 con 0x7f4f74100e70 2024-04-25T12:06:33.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:32 smithi169 ceph-mon[32710]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T12:06:33.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:32 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1673372588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:06:33.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:32 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3843833814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:06:33.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:32 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1060818490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:06:33.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:32 smithi150 ceph-mon[32250]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T12:06:33.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:32 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1673372588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:06:33.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:32 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3843833814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:06:33.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:32 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1060818490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:06:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.167+0000 7f4f78e31640 1 -- 172.21.15.19:0/3110744457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4f74062b80 con 0x7f4f74100e70 2024-04-25T12:06:33.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.167+0000 7f4f637fe640 1 -- 172.21.15.19:0/3110744457 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f4f640bce50 con 0x7f4f74100e70 2024-04-25T12:06:33.165 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T12:06:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.169+0000 7f4f78e31640 1 -- 172.21.15.19:0/3110744457 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4f5c0a3bb0 msgr2=0x7f4f5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.169+0000 7f4f78e31640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4f5c0a3bb0 0x7f4f5c0a6070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4f740699c0 tx=0x7f4f68008040 comp rx=0 tx=0).stop 2024-04-25T12:06:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.169+0000 7f4f78e31640 1 -- 172.21.15.19:0/3110744457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f74100e70 msgr2=0x7f4f74069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.169+0000 7f4f78e31640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f74100e70 0x7f4f74069c50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4f64031570 tx=0x7f4f64032d80 comp rx=0 tx=0).stop 2024-04-25T12:06:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.169+0000 7f4f78e31640 1 -- 172.21.15.19:0/3110744457 shutdown_connections 2024-04-25T12:06:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.169+0000 7f4f78e31640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f4f5c0a3bb0 0x7f4f5c0a6070 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.169+0000 7f4f78e31640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f74102980 0x7f4f7419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.169+0000 7f4f78e31640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f74101fc0 0x7f4f7419d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.169+0000 7f4f78e31640 1 --2- 172.21.15.19:0/3110744457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f74100e70 0x7f4f74069c50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.169+0000 7f4f78e31640 1 -- 172.21.15.19:0/3110744457 >> 172.21.15.19:0/3110744457 conn(0x7f4f740fc3e0 msgr2=0x7f4f741053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.169+0000 7f4f78e31640 1 -- 172.21.15.19:0/3110744457 shutdown_connections 2024-04-25T12:06:33.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.169+0000 7f4f78e31640 1 -- 172.21.15.19:0/3110744457 wait complete. 2024-04-25T12:06:33.178 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T12:06:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.285+0000 7fa4da48a640 1 -- 172.21.15.19:0/3947431330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4102a00 msgr2=0x7fa4d4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.285+0000 7fa4da48a640 1 --2- 172.21.15.19:0/3947431330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4102a00 0x7fa4d4109320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009ca0 tx=0x7fa4cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.285+0000 7fa4da48a640 1 -- 172.21.15.19:0/3947431330 shutdown_connections 2024-04-25T12:06:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.285+0000 7fa4da48a640 1 --2- 172.21.15.19:0/3947431330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4102a00 0x7fa4d4109320 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.285+0000 7fa4da48a640 1 --2- 172.21.15.19:0/3947431330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4d4102040 0x7fa4d41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.285+0000 7fa4da48a640 1 --2- 172.21.15.19:0/3947431330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d4100e60 0x7fa4d4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.285+0000 7fa4da48a640 1 -- 172.21.15.19:0/3947431330 >> 172.21.15.19:0/3947431330 conn(0x7fa4d40fc440 msgr2=0x7fa4d40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.285+0000 7fa4da48a640 1 -- 172.21.15.19:0/3947431330 shutdown_connections 2024-04-25T12:06:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.285+0000 7fa4da48a640 1 -- 172.21.15.19:0/3947431330 wait complete. 2024-04-25T12:06:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.285+0000 7fa4da48a640 1 Processor -- start 2024-04-25T12:06:33.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.285+0000 7fa4da48a640 1 -- start start 2024-04-25T12:06:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4da48a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4100e60 0x7fa4d41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4da48a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4d4102040 0x7fa4d4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4da48a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d4102a00 0x7fa4d419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4da48a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d410dc40 con 0x7fa4d4100e60 2024-04-25T12:06:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4da48a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d410dac0 con 0x7fa4d4102a00 2024-04-25T12:06:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4da48a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d410ddc0 con 0x7fa4d4102040 2024-04-25T12:06:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4d3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4100e60 0x7fa4d41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4d3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4100e60 0x7fa4d41981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36300/0 (socket says 172.21.15.19:36300) 2024-04-25T12:06:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4d3fff640 1 -- 172.21.15.19:0/213897071 learned_addr learned my addr 172.21.15.19:0/213897071 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:33.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4d37fe640 1 --2- 172.21.15.19:0/213897071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4d4102040 0x7fa4d4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4d3fff640 1 -- 172.21.15.19:0/213897071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4d4102040 msgr2=0x7fa4d4198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4d3fff640 1 --2- 172.21.15.19:0/213897071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4d4102040 0x7fa4d4198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4d3fff640 1 -- 172.21.15.19:0/213897071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d4102a00 msgr2=0x7fa4d419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4d8a27640 1 --2- 172.21.15.19:0/213897071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d4102a00 0x7fa4d419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4d3fff640 1 --2- 172.21.15.19:0/213897071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d4102a00 0x7fa4d419f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4d3fff640 1 -- 172.21.15.19:0/213897071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d419fde0 con 0x7fa4d4100e60 2024-04-25T12:06:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4d3fff640 1 --2- 172.21.15.19:0/213897071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4100e60 0x7fa4d41981c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400c970 tx=0x7fa4c400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.286+0000 7fa4d17fa640 1 -- 172.21.15.19:0/213897071 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4007c10 con 0x7fa4d4100e60 2024-04-25T12:06:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.287+0000 7fa4da48a640 1 -- 172.21.15.19:0/213897071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d41a00d0 con 0x7fa4d4100e60 2024-04-25T12:06:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.287+0000 7fa4d17fa640 1 -- 172.21.15.19:0/213897071 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fa4c4007db0 con 0x7fa4d4100e60 2024-04-25T12:06:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.287+0000 7fa4d17fa640 1 -- 172.21.15.19:0/213897071 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c400f880 con 0x7fa4d4100e60 2024-04-25T12:06:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.287+0000 7fa4da48a640 1 -- 172.21.15.19:0/213897071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d41a0620 con 0x7fa4d4100e60 2024-04-25T12:06:33.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.288+0000 7fa4da48a640 1 -- 172.21.15.19:0/213897071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d4108130 con 0x7fa4d4100e60 2024-04-25T12:06:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.289+0000 7fa4d17fa640 1 -- 172.21.15.19:0/213897071 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa4c400fa20 con 0x7fa4d4100e60 2024-04-25T12:06:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.290+0000 7fa4d17fa640 1 --2- 172.21.15.19:0/213897071 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa4bc0a3bb0 0x7fa4bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.290+0000 7fa4d37fe640 1 --2- 172.21.15.19:0/213897071 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa4bc0a3bb0 0x7fa4bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.290+0000 7fa4d17fa640 1 -- 172.21.15.19:0/213897071 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fa4c40d0a70 con 0x7fa4d4100e60 2024-04-25T12:06:33.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.290+0000 7fa4d37fe640 1 --2- 172.21.15.19:0/213897071 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa4bc0a3bb0 0x7fa4bc0a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8005fd0 tx=0x7fa4c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:33.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.293+0000 7fa4d17fa640 1 -- 172.21.15.19:0/213897071 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4c409d550 con 0x7fa4d4100e60 2024-04-25T12:06:33.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.481+0000 7fa4da48a640 1 -- 172.21.15.19:0/213897071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa4d4062b80 con 0x7fa4d4100e60 2024-04-25T12:06:33.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.482+0000 7fa4d17fa640 1 -- 172.21.15.19:0/213897071 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fa4c409ce50 con 0x7fa4d4100e60 2024-04-25T12:06:33.482 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T12:06:33.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.484+0000 7fa4da48a640 1 -- 172.21.15.19:0/213897071 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa4bc0a3bb0 msgr2=0x7fa4bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.484+0000 7fa4da48a640 1 --2- 172.21.15.19:0/213897071 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa4bc0a3bb0 0x7fa4bc0a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8005fd0 tx=0x7fa4c8005850 comp rx=0 tx=0).stop 2024-04-25T12:06:33.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.484+0000 7fa4da48a640 1 -- 172.21.15.19:0/213897071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4100e60 msgr2=0x7fa4d41981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.484+0000 7fa4da48a640 1 --2- 172.21.15.19:0/213897071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4100e60 0x7fa4d41981c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400c970 tx=0x7fa4c400ce30 comp rx=0 tx=0).stop 2024-04-25T12:06:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.484+0000 7fa4da48a640 1 -- 172.21.15.19:0/213897071 shutdown_connections 2024-04-25T12:06:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.484+0000 7fa4da48a640 1 --2- 172.21.15.19:0/213897071 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fa4bc0a3bb0 0x7fa4bc0a6070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.484+0000 7fa4da48a640 1 --2- 172.21.15.19:0/213897071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4d4102a00 0x7fa4d419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.484+0000 7fa4da48a640 1 --2- 172.21.15.19:0/213897071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4d4102040 0x7fa4d4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.484+0000 7fa4da48a640 1 --2- 172.21.15.19:0/213897071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4d4100e60 0x7fa4d41981c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.484+0000 7fa4da48a640 1 -- 172.21.15.19:0/213897071 >> 172.21.15.19:0/213897071 conn(0x7fa4d40fc440 msgr2=0x7fa4d4105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.484+0000 7fa4da48a640 1 -- 172.21.15.19:0/213897071 shutdown_connections 2024-04-25T12:06:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.484+0000 7fa4da48a640 1 -- 172.21.15.19:0/213897071 wait complete. 2024-04-25T12:06:33.493 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:06:33.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.597+0000 7f95f8d43640 1 -- 172.21.15.19:0/4258841551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f41029a0 msgr2=0x7f95f41092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.597+0000 7f95f8d43640 1 --2- 172.21.15.19:0/4258841551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f41029a0 0x7f95f41092c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009ca0 tx=0x7f95ec02a8c0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.597+0000 7f95f8d43640 1 -- 172.21.15.19:0/4258841551 shutdown_connections 2024-04-25T12:06:33.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.597+0000 7f95f8d43640 1 --2- 172.21.15.19:0/4258841551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f41029a0 0x7f95f41092c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.597+0000 7f95f8d43640 1 --2- 172.21.15.19:0/4258841551 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f4101fe0 0x7f95f4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.597+0000 7f95f8d43640 1 --2- 172.21.15.19:0/4258841551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f4100e90 0x7f95f41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.598+0000 7f95f8d43640 1 -- 172.21.15.19:0/4258841551 >> 172.21.15.19:0/4258841551 conn(0x7f95f40fc400 msgr2=0x7f95f40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:33.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.598+0000 7f95f8d43640 1 -- 172.21.15.19:0/4258841551 shutdown_connections 2024-04-25T12:06:33.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.598+0000 7f95f8d43640 1 -- 172.21.15.19:0/4258841551 wait complete. 2024-04-25T12:06:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.598+0000 7f95f8d43640 1 Processor -- start 2024-04-25T12:06:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.598+0000 7f95f8d43640 1 -- start start 2024-04-25T12:06:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.598+0000 7f95f8d43640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f4100e90 0x7f95f41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.598+0000 7f95f8d43640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f4101fe0 0x7f95f41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f8d43640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f41029a0 0x7f95f419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f8d43640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95f410cb60 con 0x7f95f4101fe0 2024-04-25T12:06:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f8d43640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f95f410c9e0 con 0x7f95f41029a0 2024-04-25T12:06:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f8d43640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f95f410cce0 con 0x7f95f4100e90 2024-04-25T12:06:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f4101fe0 0x7f95f41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f4101fe0 0x7f95f41986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36320/0 (socket says 172.21.15.19:36320) 2024-04-25T12:06:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f1d9b640 1 -- 172.21.15.19:0/2263735681 learned_addr learned my addr 172.21.15.19:0/2263735681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f259c640 1 --2- 172.21.15.19:0/2263735681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f4100e90 0x7f95f41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f1d9b640 1 -- 172.21.15.19:0/2263735681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f4100e90 msgr2=0x7f95f41981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f1d9b640 1 --2- 172.21.15.19:0/2263735681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f4100e90 0x7f95f41981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f1d9b640 1 -- 172.21.15.19:0/2263735681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f41029a0 msgr2=0x7f95f419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f2d9d640 1 --2- 172.21.15.19:0/2263735681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f41029a0 0x7f95f419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f1d9b640 1 --2- 172.21.15.19:0/2263735681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f41029a0 0x7f95f419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f1d9b640 1 -- 172.21.15.19:0/2263735681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f419fdd0 con 0x7f95f4101fe0 2024-04-25T12:06:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f1d9b640 1 --2- 172.21.15.19:0/2263735681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f4101fe0 0x7f95f41986f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f95e800c970 tx=0x7f95e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95e37fe640 1 -- 172.21.15.19:0/2263735681 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8007c10 con 0x7f95f4101fe0 2024-04-25T12:06:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95e37fe640 1 -- 172.21.15.19:0/2263735681 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f95e8007db0 con 0x7f95f4101fe0 2024-04-25T12:06:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95e37fe640 1 -- 172.21.15.19:0/2263735681 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e800f910 con 0x7f95f4101fe0 2024-04-25T12:06:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f8d43640 1 -- 172.21.15.19:0/2263735681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f41a00c0 con 0x7f95f4101fe0 2024-04-25T12:06:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.599+0000 7f95f8d43640 1 -- 172.21.15.19:0/2263735681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f41a0660 con 0x7f95f4101fe0 2024-04-25T12:06:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.600+0000 7f95f8d43640 1 -- 172.21.15.19:0/2263735681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f41a0920 con 0x7f95f4101fe0 2024-04-25T12:06:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.602+0000 7f95e37fe640 1 -- 172.21.15.19:0/2263735681 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f95e800fab0 con 0x7f95f4101fe0 2024-04-25T12:06:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.602+0000 7f95e37fe640 1 --2- 172.21.15.19:0/2263735681 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f95dc0a3c30 0x7f95dc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.602+0000 7f95f259c640 1 --2- 172.21.15.19:0/2263735681 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f95dc0a3c30 0x7f95dc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:33.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.603+0000 7f95e37fe640 1 -- 172.21.15.19:0/2263735681 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f95e80d0790 con 0x7f95f4101fe0 2024-04-25T12:06:33.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.603+0000 7f95f259c640 1 --2- 172.21.15.19:0/2263735681 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f95dc0a3c30 0x7f95dc0a60f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f95e4005fd0 tx=0x7f95e4005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:33.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.607+0000 7f95e37fe640 1 -- 172.21.15.19:0/2263735681 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95e809d1e0 con 0x7f95f4101fe0 2024-04-25T12:06:33.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:33 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/887307391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:06:33.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:33 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3110744457' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:06:33.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:33 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/213897071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:06:33.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.834+0000 7f95f8d43640 1 -- 172.21.15.19:0/2263735681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95f41a0af0 con 0x7f95f4101fe0 2024-04-25T12:06:33.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.835+0000 7f95e37fe640 1 -- 172.21.15.19:0/2263735681 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11944 (secure 0 0 0) 0x7f95e809cae0 con 0x7f95f4101fe0 2024-04-25T12:06:33.833 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:33.833 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T12:05:04.008679+0000","modified":"2024-04-25T12:05:11.186222+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24514,"mds_1":24500,"mds_2":24496,"mds_3":24484,"mds_4":14655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/1694897630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1694897630},{"type":"v1","addr":"172.21.15.19:6839","nonce":1694897630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6841/3247157172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3247157172},{"type":"v1","addr":"172.21.15.19:6841","nonce":3247157172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/3918801450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3918801450},{"type":"v1","addr":"172.21.15.150:6837","nonce":3918801450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2553528303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2553528303},{"type":"v1","addr":"172.21.15.150:6839","nonce":2553528303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/1723439500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1723439500},{"type":"v1","addr":"172.21.15.169:6839","nonce":1723439500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24500,24514,24496,24484,14655]},"id":1}]} 2024-04-25T12:06:33.833 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-25T12:06:33.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.837+0000 7f95f8d43640 1 -- 172.21.15.19:0/2263735681 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f95dc0a3c30 msgr2=0x7f95dc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.837+0000 7f95f8d43640 1 --2- 172.21.15.19:0/2263735681 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f95dc0a3c30 0x7f95dc0a60f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f95e4005fd0 tx=0x7f95e4005c70 comp rx=0 tx=0).stop 2024-04-25T12:06:33.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.837+0000 7f95f8d43640 1 -- 172.21.15.19:0/2263735681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f4101fe0 msgr2=0x7f95f41986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.837+0000 7f95f8d43640 1 --2- 172.21.15.19:0/2263735681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f4101fe0 0x7f95f41986f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f95e800c970 tx=0x7f95e800ce30 comp rx=0 tx=0).stop 2024-04-25T12:06:33.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.837+0000 7f95f8d43640 1 -- 172.21.15.19:0/2263735681 shutdown_connections 2024-04-25T12:06:33.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.837+0000 7f95f8d43640 1 --2- 172.21.15.19:0/2263735681 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f95dc0a3c30 0x7f95dc0a60f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.837+0000 7f95f8d43640 1 --2- 172.21.15.19:0/2263735681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95f41029a0 0x7f95f419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.837+0000 7f95f8d43640 1 --2- 172.21.15.19:0/2263735681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95f4101fe0 0x7f95f41986f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.837+0000 7f95f8d43640 1 --2- 172.21.15.19:0/2263735681 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95f4100e90 0x7f95f41981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.837+0000 7f95f8d43640 1 -- 172.21.15.19:0/2263735681 >> 172.21.15.19:0/2263735681 conn(0x7f95f40fc400 msgr2=0x7f95f40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:33.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.837+0000 7f95f8d43640 1 -- 172.21.15.19:0/2263735681 shutdown_connections 2024-04-25T12:06:33.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.837+0000 7f95f8d43640 1 -- 172.21.15.19:0/2263735681 wait complete. 2024-04-25T12:06:33.846 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:06:33.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.951+0000 7f0e74922640 1 -- 172.21.15.19:0/3112961564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e701031b0 msgr2=0x7f0e70103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.951+0000 7f0e74922640 1 --2- 172.21.15.19:0/3112961564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e701031b0 0x7f0e70103630 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009900 tx=0x7f0e6002a410 comp rx=0 tx=0).stop 2024-04-25T12:06:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.951+0000 7f0e74922640 1 -- 172.21.15.19:0/3112961564 shutdown_connections 2024-04-25T12:06:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.951+0000 7f0e74922640 1 --2- 172.21.15.19:0/3112961564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e70103b70 0x7f0e7010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.951+0000 7f0e74922640 1 --2- 172.21.15.19:0/3112961564 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e701031b0 0x7f0e70103630 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.951+0000 7f0e74922640 1 --2- 172.21.15.19:0/3112961564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e70101fc0 0x7f0e701023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.951+0000 7f0e74922640 1 -- 172.21.15.19:0/3112961564 >> 172.21.15.19:0/3112961564 conn(0x7f0e700fd550 msgr2=0x7f0e700ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.951+0000 7f0e74922640 1 -- 172.21.15.19:0/3112961564 shutdown_connections 2024-04-25T12:06:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.951+0000 7f0e74922640 1 -- 172.21.15.19:0/3112961564 wait complete. 2024-04-25T12:06:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e74922640 1 Processor -- start 2024-04-25T12:06:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e74922640 1 -- start start 2024-04-25T12:06:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e74922640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70101fc0 0x7f0e70198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e74922640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e701031b0 0x7f0e701987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e74922640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e70103b70 0x7f0e7019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e74922640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7010dc20 con 0x7f0e70101fc0 2024-04-25T12:06:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e74922640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7010daa0 con 0x7f0e70103b70 2024-04-25T12:06:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e74922640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7010dda0 con 0x7f0e701031b0 2024-04-25T12:06:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e6e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70101fc0 0x7f0e70198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e6e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70101fc0 0x7f0e70198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36332/0 (socket says 172.21.15.19:36332) 2024-04-25T12:06:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e6e59c640 1 -- 172.21.15.19:0/601635986 learned_addr learned my addr 172.21.15.19:0/601635986 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e6ed9d640 1 --2- 172.21.15.19:0/601635986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e70103b70 0x7f0e7019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e6dd9b640 1 --2- 172.21.15.19:0/601635986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e701031b0 0x7f0e701987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e6e59c640 1 -- 172.21.15.19:0/601635986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e701031b0 msgr2=0x7f0e701987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e6e59c640 1 --2- 172.21.15.19:0/601635986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e701031b0 0x7f0e701987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e6e59c640 1 -- 172.21.15.19:0/601635986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e70103b70 msgr2=0x7f0e7019f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e6e59c640 1 --2- 172.21.15.19:0/601635986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e70103b70 0x7f0e7019f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.952+0000 7f0e6e59c640 1 -- 172.21.15.19:0/601635986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7019fea0 con 0x7f0e70101fc0 2024-04-25T12:06:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.953+0000 7f0e6e59c640 1 --2- 172.21.15.19:0/601635986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70101fc0 0x7f0e70198280 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800e9c0 tx=0x7f0e5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.953+0000 7f0e677fe640 1 -- 172.21.15.19:0/601635986 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800cd80 con 0x7f0e70101fc0 2024-04-25T12:06:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.953+0000 7f0e74922640 1 -- 172.21.15.19:0/601635986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e701a0190 con 0x7f0e70101fc0 2024-04-25T12:06:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.953+0000 7f0e677fe640 1 -- 172.21.15.19:0/601635986 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0e580035f0 con 0x7f0e70101fc0 2024-04-25T12:06:33.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.953+0000 7f0e677fe640 1 -- 172.21.15.19:0/601635986 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58010690 con 0x7f0e70101fc0 2024-04-25T12:06:33.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.953+0000 7f0e74922640 1 -- 172.21.15.19:0/601635986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e701a0700 con 0x7f0e70101fc0 2024-04-25T12:06:33.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.954+0000 7f0e74922640 1 -- 172.21.15.19:0/601635986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e701092e0 con 0x7f0e70101fc0 2024-04-25T12:06:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.955+0000 7f0e677fe640 1 -- 172.21.15.19:0/601635986 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0e580037a0 con 0x7f0e70101fc0 2024-04-25T12:06:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.956+0000 7f0e677fe640 1 --2- 172.21.15.19:0/601635986 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f0e5c0a3bb0 0x7f0e5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.956+0000 7f0e6dd9b640 1 --2- 172.21.15.19:0/601635986 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f0e5c0a3bb0 0x7f0e5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.956+0000 7f0e677fe640 1 -- 172.21.15.19:0/601635986 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f0e580d07e0 con 0x7f0e70101fc0 2024-04-25T12:06:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.956+0000 7f0e6dd9b640 1 --2- 172.21.15.19:0/601635986 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f0e5c0a3bb0 0x7f0e5c0a6070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0e60000c00 tx=0x7f0e60031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:33.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:33.960+0000 7f0e677fe640 1 -- 172.21.15.19:0/601635986 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e5809d1a0 con 0x7f0e70101fc0 2024-04-25T12:06:34.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.149+0000 7f0e74922640 1 -- 172.21.15.19:0/601635986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e70062b50 con 0x7f0e70101fc0 2024-04-25T12:06:34.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.151+0000 7f0e677fe640 1 -- 172.21.15.19:0/601635986 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22731 (secure 0 0 0) 0x7f0e5809caa0 con 0x7f0e70101fc0 2024-04-25T12:06:34.149 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:34.149 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":87,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:05:55.975412+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:05:01.658354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:05:02.969078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:06:34.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.153+0000 7f0e74922640 1 -- 172.21.15.19:0/601635986 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f0e5c0a3bb0 msgr2=0x7f0e5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.153+0000 7f0e74922640 1 --2- 172.21.15.19:0/601635986 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f0e5c0a3bb0 0x7f0e5c0a6070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0e60000c00 tx=0x7f0e60031510 comp rx=0 tx=0).stop 2024-04-25T12:06:34.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.153+0000 7f0e74922640 1 -- 172.21.15.19:0/601635986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70101fc0 msgr2=0x7f0e70198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.153+0000 7f0e74922640 1 --2- 172.21.15.19:0/601635986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70101fc0 0x7f0e70198280 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800e9c0 tx=0x7f0e5800ee80 comp rx=0 tx=0).stop 2024-04-25T12:06:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.153+0000 7f0e74922640 1 -- 172.21.15.19:0/601635986 shutdown_connections 2024-04-25T12:06:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.153+0000 7f0e74922640 1 --2- 172.21.15.19:0/601635986 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f0e5c0a3bb0 0x7f0e5c0a6070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.153+0000 7f0e74922640 1 --2- 172.21.15.19:0/601635986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e70103b70 0x7f0e7019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.153+0000 7f0e74922640 1 --2- 172.21.15.19:0/601635986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e701031b0 0x7f0e701987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.153+0000 7f0e74922640 1 --2- 172.21.15.19:0/601635986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e70101fc0 0x7f0e70198280 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.153+0000 7f0e74922640 1 -- 172.21.15.19:0/601635986 >> 172.21.15.19:0/601635986 conn(0x7f0e700fd550 msgr2=0x7f0e701064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.153+0000 7f0e74922640 1 -- 172.21.15.19:0/601635986 shutdown_connections 2024-04-25T12:06:34.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.153+0000 7f0e74922640 1 -- 172.21.15.19:0/601635986 wait complete. 2024-04-25T12:06:34.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:33 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/887307391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:06:34.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:33 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3110744457' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:06:34.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:33 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/213897071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:06:34.163 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:06:34.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:33 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/887307391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:06:34.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:33 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3110744457' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:06:34.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:33 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/213897071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:06:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.267+0000 7f8057c59640 1 -- 172.21.15.19:0/4106516177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80501029a0 msgr2=0x7f80501092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.267+0000 7f8057c59640 1 --2- 172.21.15.19:0/4106516177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80501029a0 0x7f80501092c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8048009ca0 tx=0x7f804802a8c0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.267+0000 7f8057c59640 1 -- 172.21.15.19:0/4106516177 shutdown_connections 2024-04-25T12:06:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.267+0000 7f8057c59640 1 --2- 172.21.15.19:0/4106516177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80501029a0 0x7f80501092c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.267+0000 7f8057c59640 1 --2- 172.21.15.19:0/4106516177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8050101fe0 0x7f8050102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.267+0000 7f8057c59640 1 --2- 172.21.15.19:0/4106516177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8050100e90 0x7f80501012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.267+0000 7f8057c59640 1 -- 172.21.15.19:0/4106516177 >> 172.21.15.19:0/4106516177 conn(0x7f80500fc400 msgr2=0x7f80500fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.267+0000 7f8057c59640 1 -- 172.21.15.19:0/4106516177 shutdown_connections 2024-04-25T12:06:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.267+0000 7f8057c59640 1 -- 172.21.15.19:0/4106516177 wait complete. 2024-04-25T12:06:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f8057c59640 1 Processor -- start 2024-04-25T12:06:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f8057c59640 1 -- start start 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f8057c59640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8050100e90 0x7f8050198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f8057c59640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8050101fe0 0x7f8050198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f8057c59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80501029a0 0x7f805019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f8057c59640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f805010cb30 con 0x7f80501029a0 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f8057c59640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f805010c9b0 con 0x7f8050101fe0 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f8057c59640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f805010ccb0 con 0x7f8050100e90 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f80561f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80501029a0 0x7f805019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f80561f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80501029a0 0x7f805019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36354/0 (socket says 172.21.15.19:36354) 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f80561f6640 1 -- 172.21.15.19:0/3623065334 learned_addr learned my addr 172.21.15.19:0/3623065334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f80551f4640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8050101fe0 0x7f8050198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f80559f5640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8050100e90 0x7f8050198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f80561f6640 1 -- 172.21.15.19:0/3623065334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8050100e90 msgr2=0x7f8050198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f80561f6640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8050100e90 0x7f8050198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f80561f6640 1 -- 172.21.15.19:0/3623065334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8050101fe0 msgr2=0x7f8050198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f80561f6640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8050101fe0 0x7f8050198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f80561f6640 1 -- 172.21.15.19:0/3623065334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805019fe60 con 0x7f80501029a0 2024-04-25T12:06:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f80559f5640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8050100e90 0x7f8050198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:06:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.268+0000 7f80561f6640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80501029a0 0x7f805019f7f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8048009dc0 tx=0x7f8048031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.269+0000 7f8046ffd640 1 -- 172.21.15.19:0/3623065334 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048003bf0 con 0x7f80501029a0 2024-04-25T12:06:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.269+0000 7f8057c59640 1 -- 172.21.15.19:0/3623065334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80501a00f0 con 0x7f80501029a0 2024-04-25T12:06:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.269+0000 7f8046ffd640 1 -- 172.21.15.19:0/3623065334 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8048003d90 con 0x7f80501029a0 2024-04-25T12:06:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.269+0000 7f8046ffd640 1 -- 172.21.15.19:0/3623065334 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804802fcd0 con 0x7f80501029a0 2024-04-25T12:06:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.269+0000 7f8057c59640 1 -- 172.21.15.19:0/3623065334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80501a0690 con 0x7f80501029a0 2024-04-25T12:06:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.270+0000 7f8057c59640 1 -- 172.21.15.19:0/3623065334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80501a09b0 con 0x7f80501029a0 2024-04-25T12:06:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.271+0000 7f8046ffd640 1 -- 172.21.15.19:0/3623065334 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8048004040 con 0x7f80501029a0 2024-04-25T12:06:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.272+0000 7f8046ffd640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f803c0a3bb0 0x7f803c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.272+0000 7f80559f5640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f803c0a3bb0 0x7f803c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.272+0000 7f8046ffd640 1 -- 172.21.15.19:0/3623065334 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f80480f1cc0 con 0x7f80501029a0 2024-04-25T12:06:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.272+0000 7f80559f5640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f803c0a3bb0 0x7f803c0a6070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f804c009b50 tx=0x7f804c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.275+0000 7f8046ffd640 1 -- 172.21.15.19:0/3623065334 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80480be680 con 0x7f80501029a0 2024-04-25T12:06:34.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.503+0000 7f8057c59640 1 -- 172.21.15.19:0/3623065334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80501a0ca0 con 0x7f80501029a0 2024-04-25T12:06:34.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.504+0000 7f8046ffd640 1 -- 172.21.15.19:0/3623065334 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11944 (secure 0 0 0) 0x7f80480bdf80 con 0x7f80501029a0 2024-04-25T12:06:34.503 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:34.503 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T12:05:04.008679+0000","modified":"2024-04-25T12:05:11.186222+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24514,"mds_1":24500,"mds_2":24496,"mds_3":24484,"mds_4":14655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/1694897630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1694897630},{"type":"v1","addr":"172.21.15.19:6839","nonce":1694897630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6841/3247157172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3247157172},{"type":"v1","addr":"172.21.15.19:6841","nonce":3247157172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/3918801450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3918801450},{"type":"v1","addr":"172.21.15.150:6837","nonce":3918801450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2553528303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2553528303},{"type":"v1","addr":"172.21.15.150:6839","nonce":2553528303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/1723439500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1723439500},{"type":"v1","addr":"172.21.15.169:6839","nonce":1723439500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24500,24514,24496,24484,14655]},"id":1}]} 2024-04-25T12:06:34.503 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-25T12:06:34.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.506+0000 7f8057c59640 1 -- 172.21.15.19:0/3623065334 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f803c0a3bb0 msgr2=0x7f803c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.506+0000 7f8057c59640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f803c0a3bb0 0x7f803c0a6070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f804c009b50 tx=0x7f804c005f90 comp rx=0 tx=0).stop 2024-04-25T12:06:34.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.506+0000 7f8057c59640 1 -- 172.21.15.19:0/3623065334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80501029a0 msgr2=0x7f805019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.506+0000 7f8057c59640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80501029a0 0x7f805019f7f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8048009dc0 tx=0x7f8048031840 comp rx=0 tx=0).stop 2024-04-25T12:06:34.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.506+0000 7f8057c59640 1 -- 172.21.15.19:0/3623065334 shutdown_connections 2024-04-25T12:06:34.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.506+0000 7f8057c59640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f803c0a3bb0 0x7f803c0a6070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.506+0000 7f8057c59640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80501029a0 0x7f805019f7f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.506+0000 7f8057c59640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8050101fe0 0x7f8050198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.506+0000 7f8057c59640 1 --2- 172.21.15.19:0/3623065334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8050100e90 0x7f8050198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.506+0000 7f8057c59640 1 -- 172.21.15.19:0/3623065334 >> 172.21.15.19:0/3623065334 conn(0x7f80500fc400 msgr2=0x7f80500fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.506+0000 7f8057c59640 1 -- 172.21.15.19:0/3623065334 shutdown_connections 2024-04-25T12:06:34.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.506+0000 7f8057c59640 1 -- 172.21.15.19:0/3623065334 wait complete. 2024-04-25T12:06:34.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T12:05:04.008679+0000', 'modified': '2024-04-25T12:05:11.186222+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24514, 'mds_1': 24500, 'mds_2': 24496, 'mds_3': 24484, 'mds_4': 14655}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14643': {'gid': 14643, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/1694897630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 1694897630}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 1694897630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14655': {'gid': 14655, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.19:6841/3247157172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 3247157172}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 3247157172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6833/4176941259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6832', 'nonce': 4176941259}, {'type': 'v1', 'addr': '172.21.15.150:6833', 'nonce': 4176941259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6833/2217300694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6832', 'nonce': 2217300694}, {'type': 'v1', 'addr': '172.21.15.169:6833', 'nonce': 2217300694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6835/1405540140', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6834', 'nonce': 1405540140}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 1405540140}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6837/3918801450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 3918801450}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 3918801450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.169:6835/3316630328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 3316630328}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 3316630328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.169:6837/1253767842', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 1253767842}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 1253767842}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.150:6839/2553528303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 2553528303}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 2553528303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.169:6839/1723439500', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 1723439500}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 1723439500}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24514, 'qdb_cluster': [24500, 24514, 24496, 24484, 14655]} 2024-04-25T12:06:34.516 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T12:06:34.516 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T12:06:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.621+0000 7efd73023640 1 -- 172.21.15.19:0/878524243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c101fc0 msgr2=0x7efd6c1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.621+0000 7efd73023640 1 --2- 172.21.15.19:0/878524243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c101fc0 0x7efd6c1023e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efd60009900 tx=0x7efd6002a410 comp rx=0 tx=0).stop 2024-04-25T12:06:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.621+0000 7efd73023640 1 -- 172.21.15.19:0/878524243 shutdown_connections 2024-04-25T12:06:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.621+0000 7efd73023640 1 --2- 172.21.15.19:0/878524243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd6c103b70 0x7efd6c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.621+0000 7efd73023640 1 --2- 172.21.15.19:0/878524243 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd6c1031b0 0x7efd6c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.621+0000 7efd73023640 1 --2- 172.21.15.19:0/878524243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c101fc0 0x7efd6c1023e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.621+0000 7efd73023640 1 -- 172.21.15.19:0/878524243 >> 172.21.15.19:0/878524243 conn(0x7efd6c0fd530 msgr2=0x7efd6c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.621+0000 7efd73023640 1 -- 172.21.15.19:0/878524243 shutdown_connections 2024-04-25T12:06:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.621+0000 7efd73023640 1 -- 172.21.15.19:0/878524243 wait complete. 2024-04-25T12:06:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.621+0000 7efd73023640 1 Processor -- start 2024-04-25T12:06:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.621+0000 7efd73023640 1 -- start start 2024-04-25T12:06:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.621+0000 7efd73023640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c101fc0 0x7efd6c075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd73023640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd6c1031b0 0x7efd6c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd73023640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd6c103b70 0x7efd6c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd73023640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c10dc20 con 0x7efd6c101fc0 2024-04-25T12:06:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd73023640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c10daa0 con 0x7efd6c1031b0 2024-04-25T12:06:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd73023640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c10dda0 con 0x7efd6c103b70 2024-04-25T12:06:34.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd70dbf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c101fc0 0x7efd6c075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd70dbf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c101fc0 0x7efd6c075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36370/0 (socket says 172.21.15.19:36370) 2024-04-25T12:06:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd70dbf640 1 -- 172.21.15.19:0/760387100 learned_addr learned my addr 172.21.15.19:0/760387100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd70dbf640 1 -- 172.21.15.19:0/760387100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd6c103b70 msgr2=0x7efd6c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd70dbf640 1 --2- 172.21.15.19:0/760387100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd6c103b70 0x7efd6c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd6bfff640 1 --2- 172.21.15.19:0/760387100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd6c1031b0 0x7efd6c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd70dbf640 1 -- 172.21.15.19:0/760387100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd6c1031b0 msgr2=0x7efd6c074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd70dbf640 1 --2- 172.21.15.19:0/760387100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd6c1031b0 0x7efd6c074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd70dbf640 1 -- 172.21.15.19:0/760387100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6c075110 con 0x7efd6c101fc0 2024-04-25T12:06:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd6bfff640 1 --2- 172.21.15.19:0/760387100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd6c1031b0 0x7efd6c074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:06:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd70dbf640 1 --2- 172.21.15.19:0/760387100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c101fc0 0x7efd6c075d00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efd60006010 tx=0x7efd600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd69ffb640 1 -- 172.21.15.19:0/760387100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6003f030 con 0x7efd6c101fc0 2024-04-25T12:06:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd69ffb640 1 -- 172.21.15.19:0/760387100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7efd60003cd0 con 0x7efd6c101fc0 2024-04-25T12:06:34.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd73023640 1 -- 172.21.15.19:0/760387100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6c1a23b0 con 0x7efd6c101fc0 2024-04-25T12:06:34.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.622+0000 7efd69ffb640 1 -- 172.21.15.19:0/760387100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60035070 con 0x7efd6c101fc0 2024-04-25T12:06:34.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.623+0000 7efd73023640 1 -- 172.21.15.19:0/760387100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd6c1a2950 con 0x7efd6c101fc0 2024-04-25T12:06:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.625+0000 7efd69ffb640 1 -- 172.21.15.19:0/760387100 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efd60042030 con 0x7efd6c101fc0 2024-04-25T12:06:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.625+0000 7efd75e14740 1 Processor -- start 2024-04-25T12:06:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.625+0000 7efd75e14740 1 -- start start 2024-04-25T12:06:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.625+0000 7efd75e14740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 0x55d995684fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.625+0000 7efd75e14740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995685d50 0x55d9956861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.625+0000 7efd75e14740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995686710 0x55d99568cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.625+0000 7efd69ffb640 1 --2- 172.21.15.19:0/760387100 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7efd5c0a3c30 0x7efd5c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.625+0000 7efd75e14740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d9955e9c50 con 0x55d995685d50 2024-04-25T12:06:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.625+0000 7efd75e14740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d995586830 con 0x55d995684b80 2024-04-25T12:06:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.625+0000 7efd75e14740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d9955e5c10 con 0x55d995686710 2024-04-25T12:06:34.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.625+0000 7efd70dbf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995685d50 0x55d9956861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.625+0000 7efd715c0640 1 --2- 172.21.15.19:0/760387100 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7efd5c0a3c30 0x7efd5c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd6bfff640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995686710 0x55d99568cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd70dbf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995685d50 0x55d9956861d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36386/0 (socket says 172.21.15.19:36386) 2024-04-25T12:06:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd70dbf640 1 -- 172.21.15.19:0/2394650489 learned_addr learned my addr 172.21.15.19:0/2394650489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd69ffb640 1 -- 172.21.15.19:0/760387100 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7efd600f0af0 con 0x7efd6c101fc0 2024-04-25T12:06:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd715c0640 1 --2- 172.21.15.19:0/2394650489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 0x55d995684fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd70dbf640 1 -- 172.21.15.19:0/2394650489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995686710 msgr2=0x55d99568cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd70dbf640 1 --2- 172.21.15.19:0/2394650489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995686710 0x55d99568cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd70dbf640 1 -- 172.21.15.19:0/2394650489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 msgr2=0x55d995684fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd70dbf640 1 --2- 172.21.15.19:0/2394650489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 0x55d995684fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd70dbf640 1 -- 172.21.15.19:0/2394650489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9952fc2d0 con 0x55d995685d50 2024-04-25T12:06:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd715c0640 1 --2- 172.21.15.19:0/2394650489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 0x55d995684fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:34.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd715c0640 1 --2- 172.21.15.19:0/760387100 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7efd5c0a3c30 0x7efd5c0a60f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd64005c40 tx=0x7efd640149e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd70dbf640 1 --2- 172.21.15.19:0/2394650489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995685d50 0x55d9956861d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efd60000c00 tx=0x7efd6008e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda846e92ef16fda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd56ffd640 1 -- 172.21.15.19:0/2394650489 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd600bd4b0 con 0x55d995685d50 2024-04-25T12:06:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd56ffd640 1 -- 172.21.15.19:0/2394650489 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7efd60087910 con 0x55d995685d50 2024-04-25T12:06:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.626+0000 7efd56ffd640 1 -- 172.21.15.19:0/2394650489 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6003bd60 con 0x55d995685d50 2024-04-25T12:06:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.627+0000 7efd75e14740 1 -- 172.21.15.19:0/2394650489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995685d50 msgr2=0x55d9956861d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.627+0000 7efd75e14740 1 --2- 172.21.15.19:0/2394650489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995685d50 0x55d9956861d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efd60000c00 tx=0x7efd6008e840 comp rx=0 tx=0).stop 2024-04-25T12:06:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.627+0000 7efd75e14740 1 -- 172.21.15.19:0/2394650489 shutdown_connections 2024-04-25T12:06:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.627+0000 7efd75e14740 1 --2- 172.21.15.19:0/2394650489 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995686710 0x55d99568cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.627+0000 7efd75e14740 1 --2- 172.21.15.19:0/2394650489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995685d50 0x55d9956861d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.627+0000 7efd75e14740 1 --2- 172.21.15.19:0/2394650489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 0x55d995684fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.627+0000 7efd75e14740 1 -- 172.21.15.19:0/2394650489 >> 172.21.15.19:0/2394650489 conn(0x55d995680070 msgr2=0x55d995680470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.627+0000 7efd75e14740 1 -- 172.21.15.19:0/2394650489 shutdown_connections 2024-04-25T12:06:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.627+0000 7efd75e14740 1 -- 172.21.15.19:0/2394650489 wait complete. 2024-04-25T12:06:34.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.627+0000 7efd75e14740 1 -- start start 2024-04-25T12:06:34.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.627+0000 7efd75e14740 1 Processor -- start 2024-04-25T12:06:34.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd75e14740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 0x55d99568ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd75e14740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995685d50 0x55d99568f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd75e14740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d9956961f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd75e14740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d9955e9c50 con 0x55d995686710 2024-04-25T12:06:34.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd75e14740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d995584470 con 0x55d995684b80 2024-04-25T12:06:34.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd75e14740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d99568d4e0 con 0x55d995685d50 2024-04-25T12:06:34.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd75e14740 10 client.0 _ll_register_callbacks cb 0x55d9955a9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:06:34.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd715c0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d9956961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd715c0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d9956961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36392/0 (socket says 172.21.15.19:36392) 2024-04-25T12:06:34.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd715c0640 1 -- 172.21.15.19:0/4190296882 learned_addr learned my addr 172.21.15.19:0/4190296882 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd6bfff640 1 --2- 172.21.15.19:0/4190296882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995685d50 0x55d99568f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd70dbf640 1 --2- 172.21.15.19:0/4190296882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 0x55d99568ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd715c0640 1 -- 172.21.15.19:0/4190296882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995685d50 msgr2=0x55d99568f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd715c0640 1 --2- 172.21.15.19:0/4190296882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995685d50 0x55d99568f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd715c0640 1 -- 172.21.15.19:0/4190296882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 msgr2=0x55d99568ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd715c0640 1 --2- 172.21.15.19:0/4190296882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 0x55d99568ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd715c0640 1 -- 172.21.15.19:0/4190296882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9953c66b0 con 0x55d995686710 2024-04-25T12:06:34.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd715c0640 1 --2- 172.21.15.19:0/4190296882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d9956961f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efd640143a0 tx=0x7efd64005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5439b61de0e15baa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:06:34.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd4bfff640 1 -- 172.21.15.19:0/4190296882 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64023030 con 0x55d995686710 2024-04-25T12:06:34.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd75e14740 1 -- 172.21.15.19:0/4190296882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd60087910 con 0x55d995686710 2024-04-25T12:06:34.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.628+0000 7efd4bfff640 1 -- 172.21.15.19:0/4190296882 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7efd640127e0 con 0x55d995686710 2024-04-25T12:06:34.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.629+0000 7efd4bfff640 1 -- 172.21.15.19:0/4190296882 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efd6402e070 con 0x55d995686710 2024-04-25T12:06:34.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.629+0000 7efd75e14740 10 client.15081 fetch_fsmap learned FSMap version 26 2024-04-25T12:06:34.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.629+0000 7efd75e14740 1 -- 172.21.15.19:0/4190296882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9956995d0 con 0x55d995686710 2024-04-25T12:06:34.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.629+0000 7efd4bfff640 1 -- 172.21.15.19:0/4190296882 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7efd60087910 con 0x55d995686710 2024-04-25T12:06:34.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.629+0000 7efd4bfff640 1 -- 172.21.15.19:0/4190296882 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efd6401e040 con 0x55d995686710 2024-04-25T12:06:34.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.629+0000 7efd75e14740 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:06:34.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.629+0000 7efd75e14740 10 client.15081 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T12:06:34.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.629+0000 7efd75e14740 20 client.15081 populate_metadata read hostname 'smithi019' 2024-04-25T12:06:34.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.629+0000 7efd75e14740 1 --2- 172.21.15.19:0/4190296882 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x55d99569bcb0 0x55d99569d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.629+0000 7efd75e14740 4 client.15081 mds_command: new command op to 14655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T12:06:34.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.629+0000 7efd75e14740 1 -- 172.21.15.19:0/4190296882 --> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9955f7670 con 0x55d99569bcb0 2024-04-25T12:06:34.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.629+0000 7efd70dbf640 1 --2- 172.21.15.19:0/4190296882 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x55d99569bcb0 0x55d99569d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.629+0000 7efd4bfff640 10 client.15081 ms_handle_connect on v2:172.21.15.19:6840/3247157172 2024-04-25T12:06:34.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.629+0000 7efd70dbf640 1 --2- 172.21.15.19:0/4190296882 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x55d99569bcb0 0x55d99569d120 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8046f324e2f0452c server_cookie=b24c91dab8c7fe63 in_seq=0 out_seq=0 2024-04-25T12:06:34.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.630+0000 7efd4bfff640 1 -- 172.21.15.19:0/4190296882 <== mds.4 v2:172.21.15.19:6840/3247157172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9955f7670 con 0x55d99569bcb0 2024-04-25T12:06:34.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.630+0000 7efd4bfff640 10 client.15081 handle_command_reply: tid=0 2024-04-25T12:06:34.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.630+0000 7efd75e14740 1 client.15081 shutdown 2024-04-25T12:06:34.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.630+0000 7efd75e14740 1 -- 172.21.15.19:0/4190296882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 msgr2=0x55d9956961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.630+0000 7efd75e14740 1 --2- 172.21.15.19:0/4190296882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d9956961f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efd640143a0 tx=0x7efd64005bd0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.630+0000 7efd75e14740 1 -- 172.21.15.19:0/4190296882 shutdown_connections 2024-04-25T12:06:34.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.630+0000 7efd75e14740 1 --2- 172.21.15.19:0/4190296882 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x55d99569bcb0 0x55d99569d120 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.630+0000 7efd75e14740 1 --2- 172.21.15.19:0/4190296882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d9956961f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.630+0000 7efd75e14740 1 --2- 172.21.15.19:0/4190296882 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995685d50 0x55d99568f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.630+0000 7efd4bfff640 0 client.15081 ms_handle_reset on v2:172.21.15.19:6840/3247157172 2024-04-25T12:06:34.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.630+0000 7efd75e14740 1 --2- 172.21.15.19:0/4190296882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 0x55d99568ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.630+0000 7efd75e14740 1 -- 172.21.15.19:0/4190296882 >> 172.21.15.19:0/4190296882 conn(0x55d995680070 msgr2=0x55d99567f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.630+0000 7efd75e14740 1 -- 172.21.15.19:0/4190296882 shutdown_connections 2024-04-25T12:06:34.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.630+0000 7efd75e14740 1 -- 172.21.15.19:0/4190296882 wait complete. 2024-04-25T12:06:34.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.631+0000 7efd75e14740 20 client.15081 trim_cache size 0 max 16384 2024-04-25T12:06:34.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd75e14740 1 Processor -- start 2024-04-25T12:06:34.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd75e14740 1 -- start start 2024-04-25T12:06:34.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd75e14740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 0x55d995686130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd75e14740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995685d50 0x55d9956961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd75e14740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d99568ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd75e14740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d99569b460 con 0x55d995686710 2024-04-25T12:06:34.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd75e14740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d99569bb30 con 0x55d995684b80 2024-04-25T12:06:34.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd75e14740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d99569c200 con 0x55d995685d50 2024-04-25T12:06:34.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd715c0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d99568ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd715c0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d99568ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36396/0 (socket says 172.21.15.19:36396) 2024-04-25T12:06:34.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd715c0640 1 -- 172.21.15.19:0/702690745 learned_addr learned my addr 172.21.15.19:0/702690745 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd715c0640 1 -- 172.21.15.19:0/702690745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995685d50 msgr2=0x55d9956961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd715c0640 1 --2- 172.21.15.19:0/702690745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995685d50 0x55d9956961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd715c0640 1 -- 172.21.15.19:0/702690745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 msgr2=0x55d995686130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd715c0640 1 --2- 172.21.15.19:0/702690745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 0x55d995686130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd715c0640 1 -- 172.21.15.19:0/702690745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99568fc60 con 0x55d995686710 2024-04-25T12:06:34.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd715c0640 1 --2- 172.21.15.19:0/702690745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d99568ec70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efd64014370 tx=0x7efd64005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495eab7f2720e191 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd567fc640 1 -- 172.21.15.19:0/702690745 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64023070 con 0x55d995686710 2024-04-25T12:06:34.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd567fc640 1 -- 172.21.15.19:0/702690745 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7efd64038070 con 0x55d995686710 2024-04-25T12:06:34.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.649+0000 7efd567fc640 1 -- 172.21.15.19:0/702690745 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64012c50 con 0x55d995686710 2024-04-25T12:06:34.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.650+0000 7efd75e14740 1 -- 172.21.15.19:0/702690745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 msgr2=0x55d99568ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.650+0000 7efd75e14740 1 --2- 172.21.15.19:0/702690745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d99568ec70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efd64014370 tx=0x7efd64005fd0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.650+0000 7efd75e14740 1 -- 172.21.15.19:0/702690745 shutdown_connections 2024-04-25T12:06:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.650+0000 7efd75e14740 1 --2- 172.21.15.19:0/702690745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d99568ec70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.650+0000 7efd75e14740 1 --2- 172.21.15.19:0/702690745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995685d50 0x55d9956961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.650+0000 7efd75e14740 1 --2- 172.21.15.19:0/702690745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995684b80 0x55d995686130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.650+0000 7efd75e14740 1 -- 172.21.15.19:0/702690745 >> 172.21.15.19:0/702690745 conn(0x55d995680070 msgr2=0x55d99567b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.650+0000 7efd75e14740 1 -- 172.21.15.19:0/702690745 shutdown_connections 2024-04-25T12:06:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.650+0000 7efd75e14740 1 -- 172.21.15.19:0/702690745 wait complete. 2024-04-25T12:06:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.650+0000 7efd75e14740 1 -- start start 2024-04-25T12:06:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd75e14740 1 Processor -- start 2024-04-25T12:06:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd75e14740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995684b80 0x55d9956917f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd75e14740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995685d50 0x55d995691d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd75e14740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d995694cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd75e14740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d99569cfa0 con 0x55d995686710 2024-04-25T12:06:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd75e14740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d99569c8d0 con 0x55d995685d50 2024-04-25T12:06:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd75e14740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55d99569d670 con 0x55d995684b80 2024-04-25T12:06:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd75e14740 10 client.0 _ll_register_callbacks cb 0x7efd64009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:06:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd715c0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d995694cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd715c0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d995694cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36404/0 (socket says 172.21.15.19:36404) 2024-04-25T12:06:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd715c0640 1 -- 172.21.15.19:0/2626672544 learned_addr learned my addr 172.21.15.19:0/2626672544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd70dbf640 1 --2- 172.21.15.19:0/2626672544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995684b80 0x55d9956917f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd6bfff640 1 --2- 172.21.15.19:0/2626672544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995685d50 0x55d995691d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd715c0640 1 -- 172.21.15.19:0/2626672544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995684b80 msgr2=0x55d9956917f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd715c0640 1 --2- 172.21.15.19:0/2626672544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995684b80 0x55d9956917f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd715c0640 1 -- 172.21.15.19:0/2626672544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995685d50 msgr2=0x55d995691d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd715c0640 1 --2- 172.21.15.19:0/2626672544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995685d50 0x55d995691d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd715c0640 1 -- 172.21.15.19:0/2626672544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9956952a0 con 0x55d995686710 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd715c0640 1 --2- 172.21.15.19:0/2626672544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d995694cb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efd64009ab0 tx=0x7efd64004b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5229974f8874c65b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.651+0000 7efd4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd4bfff640 1 -- 172.21.15.19:0/2626672544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64023040 con 0x55d995686710 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd75e14740 1 -- 172.21.15.19:0/2626672544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd64038070 con 0x55d995686710 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd4bfff640 1 -- 172.21.15.19:0/2626672544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7efd6401f610 con 0x55d995686710 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd4bfff640 1 -- 172.21.15.19:0/2626672544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efd6401f850 con 0x55d995686710 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd75e14740 10 client.15087 fetch_fsmap learned FSMap version 26 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd75e14740 1 -- 172.21.15.19:0/2626672544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d995701c60 con 0x55d995686710 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd4bfff640 1 -- 172.21.15.19:0/2626672544 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7efd64038070 con 0x55d995686710 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd4bfff640 1 -- 172.21.15.19:0/2626672544 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efd6401e050 con 0x55d995686710 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd75e14740 10 client.15087 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd75e14740 10 client.15087 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd75e14740 20 client.15087 populate_metadata read hostname 'smithi019' 2024-04-25T12:06:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd75e14740 1 --2- 172.21.15.19:0/2626672544 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x55d995706310 0x55d995702e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd75e14740 4 client.15087 mds_command: new command op to 14655 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T12:06:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd75e14740 1 -- 172.21.15.19:0/2626672544 --> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] -- command(tid 0: {"prefix": "status"}) -- 0x55d9955f14e0 con 0x55d995706310 2024-04-25T12:06:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.652+0000 7efd70dbf640 1 --2- 172.21.15.19:0/2626672544 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x55d995706310 0x55d995702e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd4bfff640 10 client.15087 ms_handle_connect on v2:172.21.15.19:6840/3247157172 2024-04-25T12:06:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd70dbf640 1 --2- 172.21.15.19:0/2626672544 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x55d995706310 0x55d995702e20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da0845e8422d9ed8 server_cookie=c20bc1a69b818529 in_seq=0 out_seq=0 2024-04-25T12:06:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd4bfff640 1 -- 172.21.15.19:0/2626672544 <== mds.4 v2:172.21.15.19:6840/3247157172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d9955f14e0 con 0x55d995706310 2024-04-25T12:06:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd4bfff640 10 client.15087 handle_command_reply: tid=0 2024-04-25T12:06:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd75e14740 1 client.15087 shutdown 2024-04-25T12:06:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd75e14740 1 -- 172.21.15.19:0/2626672544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 msgr2=0x55d995694cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd75e14740 1 --2- 172.21.15.19:0/2626672544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d995694cb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efd64009ab0 tx=0x7efd64004b20 comp rx=0 tx=0).stop 2024-04-25T12:06:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd75e14740 1 -- 172.21.15.19:0/2626672544 shutdown_connections 2024-04-25T12:06:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd75e14740 1 --2- 172.21.15.19:0/2626672544 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x55d995706310 0x55d995702e20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd75e14740 1 --2- 172.21.15.19:0/2626672544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d995686710 0x55d995694cb0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd4bfff640 0 client.15087 ms_handle_reset on v2:172.21.15.19:6840/3247157172 2024-04-25T12:06:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd75e14740 1 --2- 172.21.15.19:0/2626672544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d995685d50 0x55d995691d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd75e14740 1 --2- 172.21.15.19:0/2626672544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55d995684b80 0x55d9956917f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd75e14740 1 -- 172.21.15.19:0/2626672544 >> 172.21.15.19:0/2626672544 conn(0x55d995680070 msgr2=0x55d995689840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd75e14740 1 -- 172.21.15.19:0/2626672544 shutdown_connections 2024-04-25T12:06:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd75e14740 1 -- 172.21.15.19:0/2626672544 wait complete. 2024-04-25T12:06:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.653+0000 7efd75e14740 20 client.15087 trim_cache size 0 max 16384 2024-04-25T12:06:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:06:34.658 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "be62e290-02fa-11ef-bc93-c7b262605968", 2024-04-25T12:06:34.658 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-04-25T12:06:34.658 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 14655, 2024-04-25T12:06:34.659 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T12:06:34.659 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T12:06:34.659 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T12:06:34.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 84.471020018999994, 2024-04-25T12:06:34.659 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-25T12:06:34.659 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 87, 2024-04-25T12:06:34.659 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T12:06:34.659 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 99.737095718000006 2024-04-25T12:06:34.659 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:06:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.654+0000 7efd73023640 1 -- 172.21.15.19:0/760387100 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7efd5c0a3c30 msgr2=0x7efd5c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.654+0000 7efd73023640 1 --2- 172.21.15.19:0/760387100 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7efd5c0a3c30 0x7efd5c0a60f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd64005c40 tx=0x7efd640149e0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.654+0000 7efd73023640 1 -- 172.21.15.19:0/760387100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c101fc0 msgr2=0x7efd6c075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.654+0000 7efd73023640 1 --2- 172.21.15.19:0/760387100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c101fc0 0x7efd6c075d00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efd60006010 tx=0x7efd600042f0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.654+0000 7efd73023640 1 -- 172.21.15.19:0/760387100 shutdown_connections 2024-04-25T12:06:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.654+0000 7efd73023640 1 --2- 172.21.15.19:0/760387100 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7efd5c0a3c30 0x7efd5c0a60f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.654+0000 7efd73023640 1 --2- 172.21.15.19:0/760387100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd6c103b70 0x7efd6c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.654+0000 7efd73023640 1 --2- 172.21.15.19:0/760387100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd6c1031b0 0x7efd6c074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.654+0000 7efd73023640 1 --2- 172.21.15.19:0/760387100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd6c101fc0 0x7efd6c075d00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.654+0000 7efd73023640 1 -- 172.21.15.19:0/760387100 >> 172.21.15.19:0/760387100 conn(0x7efd6c0fd530 msgr2=0x7efd6c106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.654+0000 7efd73023640 1 -- 172.21.15.19:0/760387100 shutdown_connections 2024-04-25T12:06:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.654+0000 7efd73023640 1 -- 172.21.15.19:0/760387100 wait complete. 2024-04-25T12:06:34.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T12:06:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.772+0000 7ff581775640 1 -- 172.21.15.19:0/3927597425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff57c102a00 msgr2=0x7ff57c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.772+0000 7ff581775640 1 --2- 172.21.15.19:0/3927597425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff57c102a00 0x7ff57c109320 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff574009ca0 tx=0x7ff57402a8c0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.772+0000 7ff581775640 1 -- 172.21.15.19:0/3927597425 shutdown_connections 2024-04-25T12:06:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.772+0000 7ff581775640 1 --2- 172.21.15.19:0/3927597425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff57c102a00 0x7ff57c109320 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.772+0000 7ff581775640 1 --2- 172.21.15.19:0/3927597425 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57c102040 0x7ff57c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.772+0000 7ff581775640 1 --2- 172.21.15.19:0/3927597425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff57c100e60 0x7ff57c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.772+0000 7ff581775640 1 -- 172.21.15.19:0/3927597425 >> 172.21.15.19:0/3927597425 conn(0x7ff57c0fc440 msgr2=0x7ff57c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.773+0000 7ff581775640 1 -- 172.21.15.19:0/3927597425 shutdown_connections 2024-04-25T12:06:34.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.773+0000 7ff581775640 1 -- 172.21.15.19:0/3927597425 wait complete. 2024-04-25T12:06:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.773+0000 7ff581775640 1 Processor -- start 2024-04-25T12:06:34.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff581775640 1 -- start start 2024-04-25T12:06:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff581775640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff57c100e60 0x7ff57c1a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff581775640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff57c102040 0x7ff57c1a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff581775640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57c102a00 0x7ff57c1a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff581775640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c10cb40 con 0x7ff57c102040 2024-04-25T12:06:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff581775640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c10c9c0 con 0x7ff57c100e60 2024-04-25T12:06:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff581775640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff57c10ccc0 con 0x7ff57c102a00 2024-04-25T12:06:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff57a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff57c102040 0x7ff57c1a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff57a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff57c102040 0x7ff57c1a3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36416/0 (socket says 172.21.15.19:36416) 2024-04-25T12:06:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff57a7fc640 1 -- 172.21.15.19:0/3751839833 learned_addr learned my addr 172.21.15.19:0/3751839833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff57affd640 1 --2- 172.21.15.19:0/3751839833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff57c100e60 0x7ff57c1a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff57b7fe640 1 --2- 172.21.15.19:0/3751839833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57c102a00 0x7ff57c1a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff57a7fc640 1 -- 172.21.15.19:0/3751839833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57c102a00 msgr2=0x7ff57c1a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff57a7fc640 1 --2- 172.21.15.19:0/3751839833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57c102a00 0x7ff57c1a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff57a7fc640 1 -- 172.21.15.19:0/3751839833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff57c100e60 msgr2=0x7ff57c1a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff57a7fc640 1 --2- 172.21.15.19:0/3751839833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff57c100e60 0x7ff57c1a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff57a7fc640 1 -- 172.21.15.19:0/3751839833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57c1a63d0 con 0x7ff57c102040 2024-04-25T12:06:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff57a7fc640 1 --2- 172.21.15.19:0/3751839833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff57c102040 0x7ff57c1a3580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c970 tx=0x7ff57000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.774+0000 7ff563fff640 1 -- 172.21.15.19:0/3751839833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570007bb0 con 0x7ff57c102040 2024-04-25T12:06:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.775+0000 7ff563fff640 1 -- 172.21.15.19:0/3751839833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff570007d50 con 0x7ff57c102040 2024-04-25T12:06:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.775+0000 7ff581775640 1 -- 172.21.15.19:0/3751839833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57c1a66c0 con 0x7ff57c102040 2024-04-25T12:06:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.775+0000 7ff563fff640 1 -- 172.21.15.19:0/3751839833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57000f450 con 0x7ff57c102040 2024-04-25T12:06:34.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.775+0000 7ff581775640 1 -- 172.21.15.19:0/3751839833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff57c1a6b90 con 0x7ff57c102040 2024-04-25T12:06:34.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.777+0000 7ff563fff640 1 -- 172.21.15.19:0/3751839833 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff5700046c0 con 0x7ff57c102040 2024-04-25T12:06:34.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.777+0000 7ff584485740 1 Processor -- start 2024-04-25T12:06:34.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.777+0000 7ff584485740 1 -- start start 2024-04-25T12:06:34.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.777+0000 7ff584485740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba2b80 0x5630a5ba2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.777+0000 7ff584485740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba3d50 0x5630a5ba41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.777+0000 7ff584485740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba4710 0x5630a5baafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.777+0000 7ff584485740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5630a5b07c50 con 0x5630a5ba4710 2024-04-25T12:06:34.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.777+0000 7ff584485740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5630a5aa4830 con 0x5630a5ba3d50 2024-04-25T12:06:34.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.777+0000 7ff584485740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5630a5b03c10 con 0x5630a5ba2b80 2024-04-25T12:06:34.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.777+0000 7ff563fff640 1 --2- 172.21.15.19:0/3751839833 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff5640a3bb0 0x7ff5640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff57a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba4710 0x5630a5baafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff57a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba4710 0x5630a5baafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36426/0 (socket says 172.21.15.19:36426) 2024-04-25T12:06:34.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff57a7fc640 1 -- 172.21.15.19:0/3475882748 learned_addr learned my addr 172.21.15.19:0/3475882748 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff57b7fe640 1 --2- 172.21.15.19:0/3751839833 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff5640a3bb0 0x7ff5640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff57b7fe640 1 --2- 172.21.15.19:0/3475882748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba2b80 0x5630a5ba2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff57affd640 1 --2- 172.21.15.19:0/3475882748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba3d50 0x5630a5ba41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff563fff640 1 -- 172.21.15.19:0/3751839833 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff5700d0120 con 0x7ff57c102040 2024-04-25T12:06:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff57a7fc640 1 -- 172.21.15.19:0/3475882748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba2b80 msgr2=0x5630a5ba2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff57a7fc640 1 --2- 172.21.15.19:0/3475882748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba2b80 0x5630a5ba2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff57a7fc640 1 -- 172.21.15.19:0/3475882748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba3d50 msgr2=0x5630a5ba41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff57a7fc640 1 --2- 172.21.15.19:0/3475882748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba3d50 0x5630a5ba41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff57a7fc640 1 -- 172.21.15.19:0/3475882748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630a581a2d0 con 0x5630a5ba4710 2024-04-25T12:06:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff57a7fc640 1 --2- 172.21.15.19:0/3475882748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba4710 0x5630a5baafa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff57009c190 tx=0x7ff5700049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c38b7b78e695653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff57b7fe640 1 --2- 172.21.15.19:0/3751839833 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff5640a3bb0 0x7ff5640a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff5740098d0 tx=0x7ff57402f3a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff5617fa640 1 -- 172.21.15.19:0/3475882748 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57001d070 con 0x5630a5ba4710 2024-04-25T12:06:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff5617fa640 1 -- 172.21.15.19:0/3475882748 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff5700056a0 con 0x5630a5ba4710 2024-04-25T12:06:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.778+0000 7ff5617fa640 1 -- 172.21.15.19:0/3475882748 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5700a2f80 con 0x5630a5ba4710 2024-04-25T12:06:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.779+0000 7ff584485740 1 -- 172.21.15.19:0/3475882748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba4710 msgr2=0x5630a5baafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.779+0000 7ff584485740 1 --2- 172.21.15.19:0/3475882748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba4710 0x5630a5baafa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff57009c190 tx=0x7ff5700049f0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.779+0000 7ff584485740 1 -- 172.21.15.19:0/3475882748 shutdown_connections 2024-04-25T12:06:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.779+0000 7ff584485740 1 --2- 172.21.15.19:0/3475882748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba4710 0x5630a5baafa0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.779+0000 7ff584485740 1 --2- 172.21.15.19:0/3475882748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba3d50 0x5630a5ba41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.779+0000 7ff584485740 1 --2- 172.21.15.19:0/3475882748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba2b80 0x5630a5ba2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.779+0000 7ff584485740 1 -- 172.21.15.19:0/3475882748 >> 172.21.15.19:0/3475882748 conn(0x5630a5b9e070 msgr2=0x5630a5b9e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.779+0000 7ff584485740 1 -- 172.21.15.19:0/3475882748 shutdown_connections 2024-04-25T12:06:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.779+0000 7ff584485740 1 -- 172.21.15.19:0/3475882748 wait complete. 2024-04-25T12:06:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.779+0000 7ff584485740 1 -- start start 2024-04-25T12:06:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.779+0000 7ff584485740 1 Processor -- start 2024-04-25T12:06:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff584485740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba2b80 0x5630a5bad070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff584485740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba3d50 0x5630a5bad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff584485740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba4710 0x5630a5bb4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff584485740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5630a5bab720 con 0x5630a5ba4710 2024-04-25T12:06:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff584485740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5630a5aa2470 con 0x5630a5ba2b80 2024-04-25T12:06:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff584485740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5630a5bab8a0 con 0x5630a5ba3d50 2024-04-25T12:06:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff584485740 10 client.0 _ll_register_callbacks cb 0x5630a5ac7f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:06:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff57b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba4710 0x5630a5bb4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff57b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba4710 0x5630a5bb4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36432/0 (socket says 172.21.15.19:36432) 2024-04-25T12:06:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff57b7fe640 1 -- 172.21.15.19:0/1390353124 learned_addr learned my addr 172.21.15.19:0/1390353124 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff57affd640 1 --2- 172.21.15.19:0/1390353124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba2b80 0x5630a5bad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff57a7fc640 1 --2- 172.21.15.19:0/1390353124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba3d50 0x5630a5bad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff57b7fe640 1 -- 172.21.15.19:0/1390353124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba3d50 msgr2=0x5630a5bad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff57b7fe640 1 --2- 172.21.15.19:0/1390353124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba3d50 0x5630a5bad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff57b7fe640 1 -- 172.21.15.19:0/1390353124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba2b80 msgr2=0x5630a5bad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff57b7fe640 1 --2- 172.21.15.19:0/1390353124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba2b80 0x5630a5bad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff57b7fe640 1 -- 172.21.15.19:0/1390353124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630a58e46b0 con 0x5630a5ba4710 2024-04-25T12:06:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff57b7fe640 1 --2- 172.21.15.19:0/1390353124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba4710 0x5630a5bb4580 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff574035000 tx=0x7ff5740093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3bfe91ba9ebb15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff55eff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:06:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff55eff5640 1 -- 172.21.15.19:0/1390353124 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574004440 con 0x5630a5ba4710 2024-04-25T12:06:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff55eff5640 1 -- 172.21.15.19:0/1390353124 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff5740045e0 con 0x5630a5ba4710 2024-04-25T12:06:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.780+0000 7ff584485740 1 -- 172.21.15.19:0/1390353124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5700056a0 con 0x5630a5ba4710 2024-04-25T12:06:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.781+0000 7ff55eff5640 1 -- 172.21.15.19:0/1390353124 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff5740047e0 con 0x5630a5ba4710 2024-04-25T12:06:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.781+0000 7ff584485740 10 client.15099 fetch_fsmap learned FSMap version 26 2024-04-25T12:06:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.781+0000 7ff584485740 1 -- 172.21.15.19:0/1390353124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5630a5bb7960 con 0x5630a5ba4710 2024-04-25T12:06:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.781+0000 7ff55eff5640 1 -- 172.21.15.19:0/1390353124 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff5700056a0 con 0x5630a5ba4710 2024-04-25T12:06:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.781+0000 7ff55eff5640 1 -- 172.21.15.19:0/1390353124 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff574030860 con 0x5630a5ba4710 2024-04-25T12:06:34.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.781+0000 7ff584485740 10 client.15099 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:06:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.781+0000 7ff584485740 10 client.15099 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T12:06:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.781+0000 7ff584485740 20 client.15099 populate_metadata read hostname 'smithi019' 2024-04-25T12:06:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.781+0000 7ff584485740 1 --2- 172.21.15.19:0/1390353124 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x5630a5bba010 0x5630a5bbb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.781+0000 7ff584485740 4 client.15099 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T12:06:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.781+0000 7ff584485740 1 -- 172.21.15.19:0/1390353124 --> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630a5b15670 con 0x5630a5bba010 2024-04-25T12:06:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.781+0000 7ff57affd640 1 --2- 172.21.15.19:0/1390353124 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x5630a5bba010 0x5630a5bbb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.782+0000 7ff55eff5640 10 client.15099 ms_handle_connect on v2:172.21.15.169:6834/3316630328 2024-04-25T12:06:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.782+0000 7ff57affd640 1 --2- 172.21.15.19:0/1390353124 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x5630a5bba010 0x5630a5bbb480 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f63b0f48e5e894cf server_cookie=5720b415eb9cb054 in_seq=0 out_seq=0 2024-04-25T12:06:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff55eff5640 1 -- 172.21.15.19:0/1390353124 <== mds.3 v2:172.21.15.169:6834/3316630328 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630a5b15670 con 0x5630a5bba010 2024-04-25T12:06:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff55eff5640 10 client.15099 handle_command_reply: tid=0 2024-04-25T12:06:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff584485740 1 client.15099 shutdown 2024-04-25T12:06:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff584485740 1 -- 172.21.15.19:0/1390353124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba4710 msgr2=0x5630a5bb4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff584485740 1 --2- 172.21.15.19:0/1390353124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba4710 0x5630a5bb4580 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff574035000 tx=0x7ff5740093c0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff584485740 1 -- 172.21.15.19:0/1390353124 shutdown_connections 2024-04-25T12:06:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff584485740 1 --2- 172.21.15.19:0/1390353124 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x5630a5bba010 0x5630a5bbb480 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff584485740 1 --2- 172.21.15.19:0/1390353124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba4710 0x5630a5bb4580 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff584485740 1 --2- 172.21.15.19:0/1390353124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba3d50 0x5630a5bad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff584485740 1 --2- 172.21.15.19:0/1390353124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba2b80 0x5630a5bad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff55eff5640 0 client.15099 ms_handle_reset on v2:172.21.15.169:6834/3316630328 2024-04-25T12:06:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff584485740 1 -- 172.21.15.19:0/1390353124 >> 172.21.15.19:0/1390353124 conn(0x5630a5b9e070 msgr2=0x5630a5b9d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff584485740 1 -- 172.21.15.19:0/1390353124 shutdown_connections 2024-04-25T12:06:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff584485740 1 -- 172.21.15.19:0/1390353124 wait complete. 2024-04-25T12:06:34.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.783+0000 7ff584485740 20 client.15099 trim_cache size 0 max 16384 2024-04-25T12:06:34.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:34 smithi019 ceph-mon[29213]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T12:06:34.797 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:34 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2263735681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:06:34.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:34 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/601635986' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:06:34.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:34 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3623065334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:06:34.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.801+0000 7ff584485740 1 Processor -- start 2024-04-25T12:06:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.801+0000 7ff584485740 1 -- start start 2024-04-25T12:06:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.801+0000 7ff584485740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba2b80 0x5630a5bacc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.801+0000 7ff584485740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba3d50 0x5630a5bb44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.801+0000 7ff584485740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba4710 0x5630a5bad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.801+0000 7ff584485740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5630a5bb97c0 con 0x5630a5ba3d50 2024-04-25T12:06:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.801+0000 7ff584485740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5630a5bb9e90 con 0x5630a5ba4710 2024-04-25T12:06:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.801+0000 7ff584485740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5630a5bba560 con 0x5630a5ba2b80 2024-04-25T12:06:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff57a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba3d50 0x5630a5bb44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff57a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba3d50 0x5630a5bb44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36442/0 (socket says 172.21.15.19:36442) 2024-04-25T12:06:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff57a7fc640 1 -- 172.21.15.19:0/3535652458 learned_addr learned my addr 172.21.15.19:0/3535652458 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff57affd640 1 --2- 172.21.15.19:0/3535652458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba2b80 0x5630a5bacc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff57a7fc640 1 -- 172.21.15.19:0/3535652458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba2b80 msgr2=0x5630a5bacc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff57b7fe640 1 --2- 172.21.15.19:0/3535652458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba4710 0x5630a5bad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff57a7fc640 1 --2- 172.21.15.19:0/3535652458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba2b80 0x5630a5bacc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff57a7fc640 1 -- 172.21.15.19:0/3535652458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba4710 msgr2=0x5630a5bad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff57a7fc640 1 --2- 172.21.15.19:0/3535652458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba4710 0x5630a5bad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff57a7fc640 1 -- 172.21.15.19:0/3535652458 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630a5ba37d0 con 0x5630a5ba3d50 2024-04-25T12:06:34.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff57b7fe640 1 --2- 172.21.15.19:0/3535652458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba4710 0x5630a5bad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:06:34.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff57a7fc640 1 --2- 172.21.15.19:0/3535652458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba3d50 0x5630a5bb44a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff5700988f0 tx=0x7ff57009c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176f6515c6edd40d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff560ff9640 1 -- 172.21.15.19:0/3535652458 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5700b4020 con 0x5630a5ba3d50 2024-04-25T12:06:34.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff560ff9640 1 -- 172.21.15.19:0/3535652458 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff5700a0790 con 0x5630a5ba3d50 2024-04-25T12:06:34.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.802+0000 7ff560ff9640 1 -- 172.21.15.19:0/3535652458 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57001d040 con 0x5630a5ba3d50 2024-04-25T12:06:34.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 -- 172.21.15.19:0/3535652458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba3d50 msgr2=0x5630a5bb44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 --2- 172.21.15.19:0/3535652458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba3d50 0x5630a5bb44a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff5700988f0 tx=0x7ff57009c2a0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 -- 172.21.15.19:0/3535652458 shutdown_connections 2024-04-25T12:06:34.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 --2- 172.21.15.19:0/3535652458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba4710 0x5630a5bad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 --2- 172.21.15.19:0/3535652458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba3d50 0x5630a5bb44a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 --2- 172.21.15.19:0/3535652458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba2b80 0x5630a5bacc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 -- 172.21.15.19:0/3535652458 >> 172.21.15.19:0/3535652458 conn(0x5630a5b9e070 msgr2=0x5630a5b990b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 -- 172.21.15.19:0/3535652458 shutdown_connections 2024-04-25T12:06:34.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 -- 172.21.15.19:0/3535652458 wait complete. 2024-04-25T12:06:34.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 -- start start 2024-04-25T12:06:34.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 Processor -- start 2024-04-25T12:06:34.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba2b80 0x5630a5ba9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba3d50 0x5630a5bb44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba4710 0x5630a5baa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5630a5bbb300 con 0x5630a5ba3d50 2024-04-25T12:06:34.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5630a5bbac30 con 0x5630a5ba2b80 2024-04-25T12:06:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5630a5bbb9d0 con 0x5630a5ba4710 2024-04-25T12:06:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.803+0000 7ff584485740 10 client.0 _ll_register_callbacks cb 0x7ff574009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:06:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff57a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba3d50 0x5630a5bb44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff57a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba3d50 0x5630a5bb44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36456/0 (socket says 172.21.15.19:36456) 2024-04-25T12:06:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff57a7fc640 1 -- 172.21.15.19:0/1448112439 learned_addr learned my addr 172.21.15.19:0/1448112439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff57affd640 1 --2- 172.21.15.19:0/1448112439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba2b80 0x5630a5ba9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff57b7fe640 1 --2- 172.21.15.19:0/1448112439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba4710 0x5630a5baa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff57a7fc640 1 -- 172.21.15.19:0/1448112439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba4710 msgr2=0x5630a5baa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff57a7fc640 1 --2- 172.21.15.19:0/1448112439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba4710 0x5630a5baa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff57a7fc640 1 -- 172.21.15.19:0/1448112439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba2b80 msgr2=0x5630a5ba9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff57a7fc640 1 --2- 172.21.15.19:0/1448112439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba2b80 0x5630a5ba9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff57a7fc640 1 -- 172.21.15.19:0/1448112439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630a5ba8a40 con 0x5630a5ba3d50 2024-04-25T12:06:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff57a7fc640 1 --2- 172.21.15.19:0/1448112439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba3d50 0x5630a5bb44a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff570007fd0 tx=0x7ff5700b0e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771d5dd9f5857171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff55eff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:06:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff55eff5640 1 -- 172.21.15.19:0/1448112439 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5700ba060 con 0x5630a5ba3d50 2024-04-25T12:06:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff55eff5640 1 -- 172.21.15.19:0/1448112439 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff5700b5b60 con 0x5630a5ba3d50 2024-04-25T12:06:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff584485740 1 -- 172.21.15.19:0/1448112439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5700a0790 con 0x5630a5ba3d50 2024-04-25T12:06:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff55eff5640 1 -- 172.21.15.19:0/1448112439 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff57001d070 con 0x5630a5ba3d50 2024-04-25T12:06:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff584485740 10 client.15105 fetch_fsmap learned FSMap version 26 2024-04-25T12:06:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff584485740 1 -- 172.21.15.19:0/1448112439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5630a5c203f0 con 0x5630a5ba3d50 2024-04-25T12:06:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.804+0000 7ff55eff5640 1 -- 172.21.15.19:0/1448112439 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff5700a0790 con 0x5630a5ba3d50 2024-04-25T12:06:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.805+0000 7ff55eff5640 1 -- 172.21.15.19:0/1448112439 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff57009b350 con 0x5630a5ba3d50 2024-04-25T12:06:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.805+0000 7ff584485740 10 client.15105 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:06:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.805+0000 7ff584485740 10 client.15105 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T12:06:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.805+0000 7ff584485740 20 client.15105 populate_metadata read hostname 'smithi019' 2024-04-25T12:06:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.805+0000 7ff584485740 1 --2- 172.21.15.19:0/1448112439 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x5630a5c24860 0x5630a5c23570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.805+0000 7ff584485740 4 client.15105 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T12:06:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.805+0000 7ff584485740 1 -- 172.21.15.19:0/1448112439 --> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] -- command(tid 0: {"prefix": "status"}) -- 0x5630a5b0f4e0 con 0x5630a5c24860 2024-04-25T12:06:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.805+0000 7ff57affd640 1 --2- 172.21.15.19:0/1448112439 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x5630a5c24860 0x5630a5c23570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.805+0000 7ff57affd640 1 --2- 172.21.15.19:0/1448112439 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x5630a5c24860 0x5630a5c23570 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=50ede94d232930de server_cookie=12b655d2b6a01cd7 in_seq=0 out_seq=0 2024-04-25T12:06:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.805+0000 7ff55eff5640 10 client.15105 ms_handle_connect on v2:172.21.15.169:6834/3316630328 2024-04-25T12:06:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff55eff5640 1 -- 172.21.15.19:0/1448112439 <== mds.3 v2:172.21.15.169:6834/3316630328 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5630a5b0f4e0 con 0x5630a5c24860 2024-04-25T12:06:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff55eff5640 10 client.15105 handle_command_reply: tid=0 2024-04-25T12:06:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff584485740 1 client.15105 shutdown 2024-04-25T12:06:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff584485740 1 -- 172.21.15.19:0/1448112439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba3d50 msgr2=0x5630a5bb44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff584485740 1 --2- 172.21.15.19:0/1448112439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba3d50 0x5630a5bb44a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff570007fd0 tx=0x7ff5700b0e80 comp rx=0 tx=0).stop 2024-04-25T12:06:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff584485740 1 -- 172.21.15.19:0/1448112439 shutdown_connections 2024-04-25T12:06:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff584485740 1 --2- 172.21.15.19:0/1448112439 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x5630a5c24860 0x5630a5c23570 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff584485740 1 --2- 172.21.15.19:0/1448112439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5630a5ba4710 0x5630a5baa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff55eff5640 0 client.15105 ms_handle_reset on v2:172.21.15.169:6834/3316630328 2024-04-25T12:06:34.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff584485740 1 --2- 172.21.15.19:0/1448112439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630a5ba3d50 0x5630a5bb44a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff584485740 1 --2- 172.21.15.19:0/1448112439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5630a5ba2b80 0x5630a5ba9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff584485740 1 -- 172.21.15.19:0/1448112439 >> 172.21.15.19:0/1448112439 conn(0x5630a5b9e070 msgr2=0x5630a5bacc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff584485740 1 -- 172.21.15.19:0/1448112439 shutdown_connections 2024-04-25T12:06:34.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff584485740 1 -- 172.21.15.19:0/1448112439 wait complete. 2024-04-25T12:06:34.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff584485740 20 client.15105 trim_cache size 0 max 16384 2024-04-25T12:06:34.811 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:06:34.811 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "be62e290-02fa-11ef-bc93-c7b262605968", 2024-04-25T12:06:34.811 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-04-25T12:06:34.811 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24484, 2024-04-25T12:06:34.811 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T12:06:34.812 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T12:06:34.812 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T12:06:34.812 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 85.094127267000005, 2024-04-25T12:06:34.812 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-25T12:06:34.812 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 87, 2024-04-25T12:06:34.812 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T12:06:34.812 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 114.25393170700001 2024-04-25T12:06:34.812 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:06:34.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff581775640 1 -- 172.21.15.19:0/3751839833 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff5640a3bb0 msgr2=0x7ff5640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff581775640 1 --2- 172.21.15.19:0/3751839833 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff5640a3bb0 0x7ff5640a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff5740098d0 tx=0x7ff57402f3a0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff581775640 1 -- 172.21.15.19:0/3751839833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff57c102040 msgr2=0x7ff57c1a3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.806+0000 7ff581775640 1 --2- 172.21.15.19:0/3751839833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff57c102040 0x7ff57c1a3580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c970 tx=0x7ff57000ce30 comp rx=0 tx=0).stop 2024-04-25T12:06:34.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.807+0000 7ff581775640 1 -- 172.21.15.19:0/3751839833 shutdown_connections 2024-04-25T12:06:34.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.807+0000 7ff581775640 1 --2- 172.21.15.19:0/3751839833 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff5640a3bb0 0x7ff5640a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.807+0000 7ff581775640 1 --2- 172.21.15.19:0/3751839833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff57c102a00 0x7ff57c1a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.807+0000 7ff581775640 1 --2- 172.21.15.19:0/3751839833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff57c102040 0x7ff57c1a3580 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.807+0000 7ff581775640 1 --2- 172.21.15.19:0/3751839833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff57c100e60 0x7ff57c1a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.807+0000 7ff581775640 1 -- 172.21.15.19:0/3751839833 >> 172.21.15.19:0/3751839833 conn(0x7ff57c0fc440 msgr2=0x7ff57c0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.807+0000 7ff581775640 1 -- 172.21.15.19:0/3751839833 shutdown_connections 2024-04-25T12:06:34.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.807+0000 7ff581775640 1 -- 172.21.15.19:0/3751839833 wait complete. 2024-04-25T12:06:34.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T12:06:34.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.921+0000 7f675cd43640 1 -- 172.21.15.19:0/2466533168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6758105360 msgr2=0x7f6758107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.921+0000 7f675cd43640 1 --2- 172.21.15.19:0/2466533168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6758105360 0x7f6758107750 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f674c009900 tx=0x7f674c02a410 comp rx=0 tx=0).stop 2024-04-25T12:06:34.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.921+0000 7f675cd43640 1 -- 172.21.15.19:0/2466533168 shutdown_connections 2024-04-25T12:06:34.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.921+0000 7f675cd43640 1 --2- 172.21.15.19:0/2466533168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6758107c90 0x7f675810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.921+0000 7f675cd43640 1 --2- 172.21.15.19:0/2466533168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6758105360 0x7f6758107750 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.921+0000 7f675cd43640 1 --2- 172.21.15.19:0/2466533168 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758102a30 0x7f6758104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.921+0000 7f675cd43640 1 -- 172.21.15.19:0/2466533168 >> 172.21.15.19:0/2466533168 conn(0x7f67580fc440 msgr2=0x7f67580fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.921+0000 7f675cd43640 1 -- 172.21.15.19:0/2466533168 shutdown_connections 2024-04-25T12:06:34.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.921+0000 7f675cd43640 1 -- 172.21.15.19:0/2466533168 wait complete. 2024-04-25T12:06:34.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f675cd43640 1 Processor -- start 2024-04-25T12:06:34.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f675cd43640 1 -- start start 2024-04-25T12:06:34.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f675cd43640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6758102a30 0x7f67581023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f675cd43640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758105360 0x7f67581009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f675cd43640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6758107c90 0x7f6758100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f675cd43640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f675810dc90 con 0x7f6758107c90 2024-04-25T12:06:34.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f675cd43640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f675810db10 con 0x7f6758102a30 2024-04-25T12:06:34.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f675cd43640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f675810de10 con 0x7f6758105360 2024-04-25T12:06:34.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f6756d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6758107c90 0x7f6758100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f6756d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6758107c90 0x7f6758100f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36462/0 (socket says 172.21.15.19:36462) 2024-04-25T12:06:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f6756d9d640 1 -- 172.21.15.19:0/1813678906 learned_addr learned my addr 172.21.15.19:0/1813678906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f6755d9b640 1 --2- 172.21.15.19:0/1813678906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758105360 0x7f67581009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f675659c640 1 --2- 172.21.15.19:0/1813678906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6758102a30 0x7f67581023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f6756d9d640 1 -- 172.21.15.19:0/1813678906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758105360 msgr2=0x7f67581009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f6756d9d640 1 --2- 172.21.15.19:0/1813678906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758105360 0x7f67581009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f6756d9d640 1 -- 172.21.15.19:0/1813678906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6758102a30 msgr2=0x7f67581023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f6756d9d640 1 --2- 172.21.15.19:0/1813678906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6758102a30 0x7f67581023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.922+0000 7f6756d9d640 1 -- 172.21.15.19:0/1813678906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67581017b0 con 0x7f6758107c90 2024-04-25T12:06:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.923+0000 7f6756d9d640 1 --2- 172.21.15.19:0/1813678906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6758107c90 0x7f6758100f20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f675000b970 tx=0x7f675000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.923+0000 7f67477fe640 1 -- 172.21.15.19:0/1813678906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750019070 con 0x7f6758107c90 2024-04-25T12:06:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.923+0000 7f67477fe640 1 -- 172.21.15.19:0/1813678906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f675000cb40 con 0x7f6758107c90 2024-04-25T12:06:34.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.923+0000 7f675cd43640 1 -- 172.21.15.19:0/1813678906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67581a2430 con 0x7f6758107c90 2024-04-25T12:06:34.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.923+0000 7f67477fe640 1 -- 172.21.15.19:0/1813678906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750007650 con 0x7f6758107c90 2024-04-25T12:06:34.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.923+0000 7f675cd43640 1 -- 172.21.15.19:0/1813678906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67581a2980 con 0x7f6758107c90 2024-04-25T12:06:34.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.925+0000 7f67477fe640 1 -- 172.21.15.19:0/1813678906 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f675001c070 con 0x7f6758107c90 2024-04-25T12:06:34.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.925+0000 7f675fa56740 1 Processor -- start 2024-04-25T12:06:34.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.925+0000 7f675fa56740 1 -- start start 2024-04-25T12:06:34.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f675fa56740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 0x5629a24a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f675fa56740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f67477fe640 1 --2- 172.21.15.19:0/1813678906 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f67400a3bb0 0x7f67400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f675fa56740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f675fa56740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5629a240ec50 con 0x5629a24ab710 2024-04-25T12:06:34.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f675fa56740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5629a23ab830 con 0x5629a24a9b80 2024-04-25T12:06:34.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f675fa56740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5629a240ac10 con 0x5629a24aad50 2024-04-25T12:06:34.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f6756d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f6755d9b640 1 --2- 172.21.15.19:0/1813678906 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f67400a3bb0 0x7f67400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f6755d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 0x5629a24a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f6755d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 0x5629a24a9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.19:60000/0 (socket says 172.21.15.19:60000) 2024-04-25T12:06:34.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f6756d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36476/0 (socket says 172.21.15.19:36476) 2024-04-25T12:06:34.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f6755d9b640 1 -- 172.21.15.19:0/980512355 learned_addr learned my addr 172.21.15.19:0/980512355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f67477fe640 1 -- 172.21.15.19:0/1813678906 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f67500cfb30 con 0x7f6758107c90 2024-04-25T12:06:34.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f675659c640 1 --2- 172.21.15.19:0/980512355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f6755d9b640 1 -- 172.21.15.19:0/980512355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 msgr2=0x5629a24ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f6755d9b640 1 --2- 172.21.15.19:0/980512355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24ab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f6755d9b640 1 -- 172.21.15.19:0/980512355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 msgr2=0x5629a24b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f6755d9b640 1 --2- 172.21.15.19:0/980512355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f6755d9b640 1 -- 172.21.15.19:0/980512355 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629a21212d0 con 0x5629a24a9b80 2024-04-25T12:06:34.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f6756d9d640 1 --2- 172.21.15.19:0/980512355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:06:34.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f6755d9b640 1 --2- 172.21.15.19:0/1813678906 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f67400a3bb0 0x7f67400a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f674c009900 tx=0x7f674c040000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.926+0000 7f6755d9b640 1 --2- 172.21.15.19:0/980512355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 0x5629a24a9fa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f675806b0f0 tx=0x7f674c004140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f61443c780aab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.927+0000 7f6744ff9640 1 -- 172.21.15.19:0/980512355 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c035040 con 0x5629a24a9b80 2024-04-25T12:06:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.927+0000 7f6744ff9640 1 -- 172.21.15.19:0/980512355 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f674c031b40 con 0x5629a24a9b80 2024-04-25T12:06:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.927+0000 7f6744ff9640 1 -- 172.21.15.19:0/980512355 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c032280 con 0x5629a24a9b80 2024-04-25T12:06:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.927+0000 7f675fa56740 1 -- 172.21.15.19:0/980512355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 msgr2=0x5629a24a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.927+0000 7f675fa56740 1 --2- 172.21.15.19:0/980512355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 0x5629a24a9fa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f675806b0f0 tx=0x7f674c004140 comp rx=0 tx=0).stop 2024-04-25T12:06:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.927+0000 7f675fa56740 1 -- 172.21.15.19:0/980512355 shutdown_connections 2024-04-25T12:06:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.927+0000 7f675fa56740 1 --2- 172.21.15.19:0/980512355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.927+0000 7f675fa56740 1 --2- 172.21.15.19:0/980512355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.927+0000 7f675fa56740 1 --2- 172.21.15.19:0/980512355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 0x5629a24a9fa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.927+0000 7f675fa56740 1 -- 172.21.15.19:0/980512355 >> 172.21.15.19:0/980512355 conn(0x5629a24a5070 msgr2=0x5629a24a5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.927+0000 7f675fa56740 1 -- 172.21.15.19:0/980512355 shutdown_connections 2024-04-25T12:06:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.927+0000 7f675fa56740 1 -- 172.21.15.19:0/980512355 wait complete. 2024-04-25T12:06:34.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675fa56740 1 -- start start 2024-04-25T12:06:34.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675fa56740 1 Processor -- start 2024-04-25T12:06:34.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675fa56740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24a9b80 0x5629a24b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675fa56740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675fa56740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24ab710 0x5629a24bb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675fa56740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5629a24b26c0 con 0x5629a24a9b80 2024-04-25T12:06:34.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675fa56740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5629a23a9470 con 0x5629a24ab710 2024-04-25T12:06:34.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675fa56740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5629a24b2840 con 0x5629a24aad50 2024-04-25T12:06:34.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675fa56740 10 client.0 _ll_register_callbacks cb 0x5629a23cef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:06:34.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24a9b80 0x5629a24b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24a9b80 0x5629a24b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36492/0 (socket says 172.21.15.19:36492) 2024-04-25T12:06:34.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675659c640 1 -- 172.21.15.19:0/1394081605 learned_addr learned my addr 172.21.15.19:0/1394081605 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f6756d9d640 1 --2- 172.21.15.19:0/1394081605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24ab710 0x5629a24bb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f6755d9b640 1 --2- 172.21.15.19:0/1394081605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675659c640 1 -- 172.21.15.19:0/1394081605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 msgr2=0x5629a24b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675659c640 1 --2- 172.21.15.19:0/1394081605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675659c640 1 -- 172.21.15.19:0/1394081605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24ab710 msgr2=0x5629a24bb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675659c640 1 --2- 172.21.15.19:0/1394081605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24ab710 0x5629a24bb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.928+0000 7f675659c640 1 -- 172.21.15.19:0/1394081605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629a21eb6b0 con 0x5629a24a9b80 2024-04-25T12:06:34.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f675659c640 1 --2- 172.21.15.19:0/1394081605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24a9b80 0x5629a24b3fe0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f674800bed0 tx=0x7f67480117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79c56763372bd59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f673dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:06:34.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f673dffb640 1 -- 172.21.15.19:0/1394081605 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748012650 con 0x5629a24a9b80 2024-04-25T12:06:34.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f673dffb640 1 -- 172.21.15.19:0/1394081605 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f67480127f0 con 0x5629a24a9b80 2024-04-25T12:06:34.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f675fa56740 1 -- 172.21.15.19:0/1394081605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f674c031b40 con 0x5629a24a9b80 2024-04-25T12:06:34.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f673dffb640 1 -- 172.21.15.19:0/1394081605 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f67480129f0 con 0x5629a24a9b80 2024-04-25T12:06:34.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f675fa56740 10 client.15117 fetch_fsmap learned FSMap version 26 2024-04-25T12:06:34.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f675fa56740 1 -- 172.21.15.19:0/1394081605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5629a24be9a0 con 0x5629a24a9b80 2024-04-25T12:06:34.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f673dffb640 1 -- 172.21.15.19:0/1394081605 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f674c031b40 con 0x5629a24a9b80 2024-04-25T12:06:34.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f673dffb640 1 -- 172.21.15.19:0/1394081605 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6748025420 con 0x5629a24a9b80 2024-04-25T12:06:34.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f675fa56740 10 client.15117 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:06:34.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f675fa56740 10 client.15117 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T12:06:34.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f675fa56740 20 client.15117 populate_metadata read hostname 'smithi019' 2024-04-25T12:06:34.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f675fa56740 1 --2- 172.21.15.19:0/1394081605 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5629a24c1010 0x5629a24c2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f675fa56740 4 client.15117 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T12:06:34.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.929+0000 7f675fa56740 1 -- 172.21.15.19:0/1394081605 --> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629a241c670 con 0x5629a24c1010 2024-04-25T12:06:34.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.930+0000 7f6755d9b640 1 --2- 172.21.15.19:0/1394081605 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5629a24c1010 0x5629a24c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.930+0000 7f673dffb640 10 client.15117 ms_handle_connect on v2:172.21.15.169:6836/1253767842 2024-04-25T12:06:34.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.930+0000 7f6755d9b640 1 --2- 172.21.15.19:0/1394081605 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5629a24c1010 0x5629a24c2480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16a9baade7975932 server_cookie=66f6553550d0722d in_seq=0 out_seq=0 2024-04-25T12:06:34.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.931+0000 7f673dffb640 1 -- 172.21.15.19:0/1394081605 <== mds.2 v2:172.21.15.169:6836/1253767842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629a241c670 con 0x5629a24c1010 2024-04-25T12:06:34.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.931+0000 7f673dffb640 10 client.15117 handle_command_reply: tid=0 2024-04-25T12:06:34.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.931+0000 7f675fa56740 1 client.15117 shutdown 2024-04-25T12:06:34.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.931+0000 7f675fa56740 1 -- 172.21.15.19:0/1394081605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24a9b80 msgr2=0x5629a24b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.931+0000 7f675fa56740 1 --2- 172.21.15.19:0/1394081605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24a9b80 0x5629a24b3fe0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f674800bed0 tx=0x7f67480117b0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.931+0000 7f675fa56740 1 -- 172.21.15.19:0/1394081605 shutdown_connections 2024-04-25T12:06:34.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.931+0000 7f675fa56740 1 --2- 172.21.15.19:0/1394081605 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5629a24c1010 0x5629a24c2480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.931+0000 7f675fa56740 1 --2- 172.21.15.19:0/1394081605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24ab710 0x5629a24bb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.931+0000 7f673dffb640 0 client.15117 ms_handle_reset on v2:172.21.15.169:6836/1253767842 2024-04-25T12:06:34.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.931+0000 7f675fa56740 1 --2- 172.21.15.19:0/1394081605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.931+0000 7f675fa56740 1 --2- 172.21.15.19:0/1394081605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24a9b80 0x5629a24b3fe0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.931+0000 7f675fa56740 1 -- 172.21.15.19:0/1394081605 >> 172.21.15.19:0/1394081605 conn(0x5629a24a5070 msgr2=0x5629a24a41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.932+0000 7f675fa56740 1 -- 172.21.15.19:0/1394081605 shutdown_connections 2024-04-25T12:06:34.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.932+0000 7f675fa56740 1 -- 172.21.15.19:0/1394081605 wait complete. 2024-04-25T12:06:34.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.932+0000 7f675fa56740 20 client.15117 trim_cache size 0 max 16384 2024-04-25T12:06:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f675fa56740 1 Processor -- start 2024-04-25T12:06:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f675fa56740 1 -- start start 2024-04-25T12:06:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f675fa56740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 0x5629a24ab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f675fa56740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24bb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f675fa56740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f675fa56740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5629a24c07c0 con 0x5629a24ab710 2024-04-25T12:06:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f675fa56740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5629a24c0e90 con 0x5629a24a9b80 2024-04-25T12:06:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f675fa56740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5629a24c1560 con 0x5629a24aad50 2024-04-25T12:06:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f6756d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f6756d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36504/0 (socket says 172.21.15.19:36504) 2024-04-25T12:06:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f6756d9d640 1 -- 172.21.15.19:0/1671949416 learned_addr learned my addr 172.21.15.19:0/1671949416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f6755d9b640 1 --2- 172.21.15.19:0/1671949416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24bb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f675659c640 1 --2- 172.21.15.19:0/1671949416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 0x5629a24ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f6756d9d640 1 -- 172.21.15.19:0/1671949416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 msgr2=0x5629a24bb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f6756d9d640 1 --2- 172.21.15.19:0/1671949416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24bb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f6756d9d640 1 -- 172.21.15.19:0/1671949416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 msgr2=0x5629a24ab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f6756d9d640 1 --2- 172.21.15.19:0/1671949416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 0x5629a24ab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f6756d9d640 1 -- 172.21.15.19:0/1671949416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629a24b5110 con 0x5629a24ab710 2024-04-25T12:06:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.950+0000 7f6756d9d640 1 --2- 172.21.15.19:0/1671949416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24a00b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f675009bd20 tx=0x7f6750007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d162d6d81a8ec1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.951+0000 7f673ffff640 1 -- 172.21.15.19:0/1671949416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675000f160 con 0x5629a24ab710 2024-04-25T12:06:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.951+0000 7f673ffff640 1 -- 172.21.15.19:0/1671949416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f675009c490 con 0x5629a24ab710 2024-04-25T12:06:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.951+0000 7f673ffff640 1 -- 172.21.15.19:0/1671949416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675001e440 con 0x5629a24ab710 2024-04-25T12:06:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.951+0000 7f675fa56740 1 -- 172.21.15.19:0/1671949416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 msgr2=0x5629a24a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.951+0000 7f675fa56740 1 --2- 172.21.15.19:0/1671949416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24a00b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f675009bd20 tx=0x7f6750007ef0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.951+0000 7f675fa56740 1 -- 172.21.15.19:0/1671949416 shutdown_connections 2024-04-25T12:06:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.951+0000 7f675fa56740 1 --2- 172.21.15.19:0/1671949416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24a00b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.951+0000 7f675fa56740 1 --2- 172.21.15.19:0/1671949416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24bb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.951+0000 7f675fa56740 1 --2- 172.21.15.19:0/1671949416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 0x5629a24ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.951+0000 7f675fa56740 1 -- 172.21.15.19:0/1671949416 >> 172.21.15.19:0/1671949416 conn(0x5629a24a5070 msgr2=0x5629a24add50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.951+0000 7f675fa56740 1 -- 172.21.15.19:0/1671949416 shutdown_connections 2024-04-25T12:06:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.951+0000 7f675fa56740 1 -- 172.21.15.19:0/1671949416 wait complete. 2024-04-25T12:06:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.951+0000 7f675fa56740 1 -- start start 2024-04-25T12:06:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f675fa56740 1 Processor -- start 2024-04-25T12:06:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f675fa56740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 0x5629a24b7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f675fa56740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24b99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f675fa56740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24b9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f675fa56740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5629a24c2300 con 0x5629a24ab710 2024-04-25T12:06:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f675fa56740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5629a24c1c30 con 0x5629a24a9b80 2024-04-25T12:06:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f675fa56740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5629a24c29d0 con 0x5629a24aad50 2024-04-25T12:06:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f675fa56740 10 client.0 _ll_register_callbacks cb 0x7f674800b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:06:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f6756d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24b9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f6756d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24b9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36514/0 (socket says 172.21.15.19:36514) 2024-04-25T12:06:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f6756d9d640 1 -- 172.21.15.19:0/3418800699 learned_addr learned my addr 172.21.15.19:0/3418800699 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f6755d9b640 1 --2- 172.21.15.19:0/3418800699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24b99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f675659c640 1 --2- 172.21.15.19:0/3418800699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 0x5629a24b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f6756d9d640 1 -- 172.21.15.19:0/3418800699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 msgr2=0x5629a24b99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f6756d9d640 1 --2- 172.21.15.19:0/3418800699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24b99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f6756d9d640 1 -- 172.21.15.19:0/3418800699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 msgr2=0x5629a24b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f6756d9d640 1 --2- 172.21.15.19:0/3418800699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 0x5629a24b7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f6756d9d640 1 -- 172.21.15.19:0/3418800699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629a24ba810 con 0x5629a24ab710 2024-04-25T12:06:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f6756d9d640 1 --2- 172.21.15.19:0/3418800699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24b9fe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f67500981f0 tx=0x7f6750004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffab6b6e9f8c7f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f673dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:06:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f673dffb640 1 -- 172.21.15.19:0/3418800699 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67500b5020 con 0x5629a24ab710 2024-04-25T12:06:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.952+0000 7f673dffb640 1 -- 172.21.15.19:0/3418800699 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f67500b51c0 con 0x5629a24ab710 2024-04-25T12:06:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.953+0000 7f675fa56740 1 -- 172.21.15.19:0/3418800699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f675009c490 con 0x5629a24ab710 2024-04-25T12:06:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.953+0000 7f673dffb640 1 -- 172.21.15.19:0/3418800699 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f675009bd90 con 0x5629a24ab710 2024-04-25T12:06:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.953+0000 7f675fa56740 10 client.15123 fetch_fsmap learned FSMap version 26 2024-04-25T12:06:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.953+0000 7f675fa56740 1 -- 172.21.15.19:0/3418800699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5629a2526fc0 con 0x5629a24ab710 2024-04-25T12:06:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.953+0000 7f673dffb640 1 -- 172.21.15.19:0/3418800699 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f675009c490 con 0x5629a24ab710 2024-04-25T12:06:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.953+0000 7f673dffb640 1 -- 172.21.15.19:0/3418800699 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6750097ae0 con 0x5629a24ab710 2024-04-25T12:06:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.953+0000 7f675fa56740 10 client.15123 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:06:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.953+0000 7f675fa56740 10 client.15123 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T12:06:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.953+0000 7f675fa56740 20 client.15123 populate_metadata read hostname 'smithi019' 2024-04-25T12:06:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.953+0000 7f675fa56740 1 --2- 172.21.15.19:0/3418800699 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5629a252b6b0 0x5629a252a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.953+0000 7f675fa56740 4 client.15123 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T12:06:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.953+0000 7f675fa56740 1 -- 172.21.15.19:0/3418800699 --> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] -- command(tid 0: {"prefix": "status"}) -- 0x5629a24164e0 con 0x5629a252b6b0 2024-04-25T12:06:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.953+0000 7f675659c640 1 --2- 172.21.15.19:0/3418800699 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5629a252b6b0 0x5629a252a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f673dffb640 10 client.15123 ms_handle_connect on v2:172.21.15.169:6836/1253767842 2024-04-25T12:06:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f675659c640 1 --2- 172.21.15.19:0/3418800699 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5629a252b6b0 0x5629a252a3c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c660a22a4aedba1d server_cookie=e47c4dd727c741a7 in_seq=0 out_seq=0 2024-04-25T12:06:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f673dffb640 1 -- 172.21.15.19:0/3418800699 <== mds.2 v2:172.21.15.169:6836/1253767842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5629a24164e0 con 0x5629a252b6b0 2024-04-25T12:06:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f673dffb640 10 client.15123 handle_command_reply: tid=0 2024-04-25T12:06:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f675fa56740 1 client.15123 shutdown 2024-04-25T12:06:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f675fa56740 1 -- 172.21.15.19:0/3418800699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 msgr2=0x5629a24b9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f675fa56740 1 --2- 172.21.15.19:0/3418800699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24b9fe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f67500981f0 tx=0x7f6750004a40 comp rx=0 tx=0).stop 2024-04-25T12:06:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f675fa56740 1 -- 172.21.15.19:0/3418800699 shutdown_connections 2024-04-25T12:06:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f675fa56740 1 --2- 172.21.15.19:0/3418800699 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5629a252b6b0 0x5629a252a3c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f675fa56740 1 --2- 172.21.15.19:0/3418800699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5629a24ab710 0x5629a24b9fe0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f675fa56740 1 --2- 172.21.15.19:0/3418800699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5629a24aad50 0x5629a24b99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f673dffb640 0 client.15123 ms_handle_reset on v2:172.21.15.169:6836/1253767842 2024-04-25T12:06:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f675fa56740 1 --2- 172.21.15.19:0/3418800699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629a24a9b80 0x5629a24b7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f675fa56740 1 -- 172.21.15.19:0/3418800699 >> 172.21.15.19:0/3418800699 conn(0x5629a24a5070 msgr2=0x5629a24a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f675fa56740 1 -- 172.21.15.19:0/3418800699 shutdown_connections 2024-04-25T12:06:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f675fa56740 1 -- 172.21.15.19:0/3418800699 wait complete. 2024-04-25T12:06:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.954+0000 7f675fa56740 20 client.15123 trim_cache size 0 max 16384 2024-04-25T12:06:34.959 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:06:34.960 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "be62e290-02fa-11ef-bc93-c7b262605968", 2024-04-25T12:06:34.960 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-04-25T12:06:34.960 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24496, 2024-04-25T12:06:34.960 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T12:06:34.960 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T12:06:34.960 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T12:06:34.960 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 86.255543493999994, 2024-04-25T12:06:34.960 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-25T12:06:34.960 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 87, 2024-04-25T12:06:34.960 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T12:06:34.960 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 111.579765815 2024-04-25T12:06:34.960 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:06:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.955+0000 7f675cd43640 1 -- 172.21.15.19:0/1813678906 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f67400a3bb0 msgr2=0x7f67400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.955+0000 7f675cd43640 1 --2- 172.21.15.19:0/1813678906 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f67400a3bb0 0x7f67400a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f674c009900 tx=0x7f674c040000 comp rx=0 tx=0).stop 2024-04-25T12:06:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.955+0000 7f675cd43640 1 -- 172.21.15.19:0/1813678906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6758107c90 msgr2=0x7f6758100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.955+0000 7f675cd43640 1 --2- 172.21.15.19:0/1813678906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6758107c90 0x7f6758100f20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f675000b970 tx=0x7f675000be30 comp rx=0 tx=0).stop 2024-04-25T12:06:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.955+0000 7f675cd43640 1 -- 172.21.15.19:0/1813678906 shutdown_connections 2024-04-25T12:06:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.955+0000 7f675cd43640 1 --2- 172.21.15.19:0/1813678906 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f67400a3bb0 0x7f67400a6070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.955+0000 7f675cd43640 1 --2- 172.21.15.19:0/1813678906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6758107c90 0x7f6758100f20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.955+0000 7f675cd43640 1 --2- 172.21.15.19:0/1813678906 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6758105360 0x7f67581009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.955+0000 7f675cd43640 1 --2- 172.21.15.19:0/1813678906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6758102a30 0x7f67581023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.955+0000 7f675cd43640 1 -- 172.21.15.19:0/1813678906 >> 172.21.15.19:0/1813678906 conn(0x7f67580fc440 msgr2=0x7f6758103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.955+0000 7f675cd43640 1 -- 172.21.15.19:0/1813678906 shutdown_connections 2024-04-25T12:06:34.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:34.955+0000 7f675cd43640 1 -- 172.21.15.19:0/1813678906 wait complete. 2024-04-25T12:06:34.963 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T12:06:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.068+0000 7fc766f51640 1 -- 172.21.15.19:0/2536985348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc760102a30 msgr2=0x7fc760104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.068+0000 7fc766f51640 1 --2- 172.21.15.19:0/2536985348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc760102a30 0x7fc760104e20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc750009900 tx=0x7fc75002a410 comp rx=0 tx=0).stop 2024-04-25T12:06:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.068+0000 7fc766f51640 1 -- 172.21.15.19:0/2536985348 shutdown_connections 2024-04-25T12:06:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.068+0000 7fc766f51640 1 --2- 172.21.15.19:0/2536985348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc760107c90 0x7fc76010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.068+0000 7fc766f51640 1 --2- 172.21.15.19:0/2536985348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc760105360 0x7fc760107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.068+0000 7fc766f51640 1 --2- 172.21.15.19:0/2536985348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc760102a30 0x7fc760104e20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.068+0000 7fc766f51640 1 -- 172.21.15.19:0/2536985348 >> 172.21.15.19:0/2536985348 conn(0x7fc7600fc440 msgr2=0x7fc7600fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.068+0000 7fc766f51640 1 -- 172.21.15.19:0/2536985348 shutdown_connections 2024-04-25T12:06:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.068+0000 7fc766f51640 1 -- 172.21.15.19:0/2536985348 wait complete. 2024-04-25T12:06:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc766f51640 1 Processor -- start 2024-04-25T12:06:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc766f51640 1 -- start start 2024-04-25T12:06:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc766f51640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc760102a30 0x7fc7601023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc766f51640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc760105360 0x7fc7601009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc766f51640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc760107c90 0x7fc760100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc766f51640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc76010dc10 con 0x7fc760102a30 2024-04-25T12:06:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc766f51640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc76010da90 con 0x7fc760107c90 2024-04-25T12:06:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc766f51640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc76010dd90 con 0x7fc760105360 2024-04-25T12:06:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc764ced640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc760102a30 0x7fc7601023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc764ced640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc760102a30 0x7fc7601023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36540/0 (socket says 172.21.15.19:36540) 2024-04-25T12:06:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc764ced640 1 -- 172.21.15.19:0/3749477599 learned_addr learned my addr 172.21.15.19:0/3749477599 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc7654ee640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc760107c90 0x7fc760100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc75ffff640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc760105360 0x7fc7601009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc764ced640 1 -- 172.21.15.19:0/3749477599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc760105360 msgr2=0x7fc7601009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc764ced640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc760105360 0x7fc7601009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc764ced640 1 -- 172.21.15.19:0/3749477599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc760107c90 msgr2=0x7fc760100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc764ced640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc760107c90 0x7fc760100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc764ced640 1 -- 172.21.15.19:0/3749477599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7601017b0 con 0x7fc760102a30 2024-04-25T12:06:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.069+0000 7fc75ffff640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc760105360 0x7fc7601009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.070+0000 7fc764ced640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc760102a30 0x7fc7601023a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc750005940 tx=0x7fc7500043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.070+0000 7fc75dffb640 1 -- 172.21.15.19:0/3749477599 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75003f030 con 0x7fc760102a30 2024-04-25T12:06:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.070+0000 7fc75dffb640 1 -- 172.21.15.19:0/3749477599 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc750003cd0 con 0x7fc760102a30 2024-04-25T12:06:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.070+0000 7fc766f51640 1 -- 172.21.15.19:0/3749477599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7601a2350 con 0x7fc760102a30 2024-04-25T12:06:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.070+0000 7fc75dffb640 1 -- 172.21.15.19:0/3749477599 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750035070 con 0x7fc760102a30 2024-04-25T12:06:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.070+0000 7fc766f51640 1 -- 172.21.15.19:0/3749477599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7601a28f0 con 0x7fc760102a30 2024-04-25T12:06:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.072+0000 7fc75dffb640 1 -- 172.21.15.19:0/3749477599 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc750042030 con 0x7fc760102a30 2024-04-25T12:06:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.072+0000 7fc769c6a740 1 Processor -- start 2024-04-25T12:06:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.072+0000 7fc769c6a740 1 -- start start 2024-04-25T12:06:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc769c6a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa49b80 0x56512aa49fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc769c6a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4ad50 0x56512aa4b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc769c6a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa4b710 0x56512aa51fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc769c6a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56512a9aec50 con 0x56512aa4ad50 2024-04-25T12:06:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc769c6a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56512a94b830 con 0x56512aa49b80 2024-04-25T12:06:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc769c6a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56512a9aac10 con 0x56512aa4b710 2024-04-25T12:06:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc75dffb640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc7480a3c30 0x7fc7480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc764ced640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4ad50 0x56512aa4b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc764ced640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4ad50 0x56512aa4b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36544/0 (socket says 172.21.15.19:36544) 2024-04-25T12:06:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc764ced640 1 -- 172.21.15.19:0/612472648 learned_addr learned my addr 172.21.15.19:0/612472648 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc7654ee640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc7480a3c30 0x7fc7480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc7654ee640 1 --2- 172.21.15.19:0/612472648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa49b80 0x56512aa49fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc75ffff640 1 --2- 172.21.15.19:0/612472648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa4b710 0x56512aa51fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc75dffb640 1 -- 172.21.15.19:0/3749477599 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc7500f09b0 con 0x7fc760102a30 2024-04-25T12:06:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc764ced640 1 -- 172.21.15.19:0/612472648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa4b710 msgr2=0x56512aa51fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc764ced640 1 --2- 172.21.15.19:0/612472648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa4b710 0x56512aa51fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc764ced640 1 -- 172.21.15.19:0/612472648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa49b80 msgr2=0x56512aa49fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc764ced640 1 --2- 172.21.15.19:0/612472648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa49b80 0x56512aa49fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc764ced640 1 -- 172.21.15.19:0/612472648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56512a6c12d0 con 0x56512aa4ad50 2024-04-25T12:06:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc764ced640 1 --2- 172.21.15.19:0/612472648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4ad50 0x56512aa4b1d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc7500b9640 tx=0x7fc75002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75515a4a0ab9b7f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc7654ee640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc7480a3c30 0x7fc7480a60f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc7580098b0 tx=0x7fc758009d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.073+0000 7fc746ffd640 1 -- 172.21.15.19:0/612472648 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7500f0e60 con 0x56512aa4ad50 2024-04-25T12:06:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.074+0000 7fc746ffd640 1 -- 172.21.15.19:0/612472648 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc7500bd370 con 0x56512aa4ad50 2024-04-25T12:06:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.074+0000 7fc746ffd640 1 -- 172.21.15.19:0/612472648 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750049070 con 0x56512aa4ad50 2024-04-25T12:06:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.074+0000 7fc769c6a740 1 -- 172.21.15.19:0/612472648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4ad50 msgr2=0x56512aa4b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.074+0000 7fc769c6a740 1 --2- 172.21.15.19:0/612472648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4ad50 0x56512aa4b1d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc7500b9640 tx=0x7fc75002a410 comp rx=0 tx=0).stop 2024-04-25T12:06:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.074+0000 7fc769c6a740 1 -- 172.21.15.19:0/612472648 shutdown_connections 2024-04-25T12:06:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.074+0000 7fc769c6a740 1 --2- 172.21.15.19:0/612472648 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa4b710 0x56512aa51fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.074+0000 7fc769c6a740 1 --2- 172.21.15.19:0/612472648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4ad50 0x56512aa4b1d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.074+0000 7fc769c6a740 1 --2- 172.21.15.19:0/612472648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa49b80 0x56512aa49fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.074+0000 7fc769c6a740 1 -- 172.21.15.19:0/612472648 >> 172.21.15.19:0/612472648 conn(0x56512aa45070 msgr2=0x56512aa45470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.074+0000 7fc769c6a740 1 -- 172.21.15.19:0/612472648 shutdown_connections 2024-04-25T12:06:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.074+0000 7fc769c6a740 1 -- 172.21.15.19:0/612472648 wait complete. 2024-04-25T12:06:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.074+0000 7fc769c6a740 1 -- start start 2024-04-25T12:06:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc769c6a740 1 Processor -- start 2024-04-25T12:06:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc769c6a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa49b80 0x56512aa53d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc769c6a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4ad50 0x56512aa54250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc769c6a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4b710 0x56512aa5b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc769c6a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56512aa524e0 con 0x56512aa4ad50 2024-04-25T12:06:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc769c6a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56512a949470 con 0x56512aa4b710 2024-04-25T12:06:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc769c6a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56512aa52660 con 0x56512aa49b80 2024-04-25T12:06:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc769c6a740 10 client.0 _ll_register_callbacks cb 0x56512a96ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:06:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc75ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4ad50 0x56512aa54250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc75ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4ad50 0x56512aa54250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36556/0 (socket says 172.21.15.19:36556) 2024-04-25T12:06:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc75ffff640 1 -- 172.21.15.19:0/3478892246 learned_addr learned my addr 172.21.15.19:0/3478892246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc764ced640 1 --2- 172.21.15.19:0/3478892246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa49b80 0x56512aa53d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc7654ee640 1 --2- 172.21.15.19:0/3478892246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4b710 0x56512aa5b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc75ffff640 1 -- 172.21.15.19:0/3478892246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa49b80 msgr2=0x56512aa53d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc75ffff640 1 --2- 172.21.15.19:0/3478892246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa49b80 0x56512aa53d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc75ffff640 1 -- 172.21.15.19:0/3478892246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4b710 msgr2=0x56512aa5b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc75ffff640 1 --2- 172.21.15.19:0/3478892246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4b710 0x56512aa5b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.075+0000 7fc75ffff640 1 -- 172.21.15.19:0/3478892246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56512a78b6b0 con 0x56512aa4ad50 2024-04-25T12:06:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc75ffff640 1 --2- 172.21.15.19:0/3478892246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4ad50 0x56512aa54250 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc75400bad0 tx=0x7fc75400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6008b02e34ac220c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc73ffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:06:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc73ffff640 1 -- 172.21.15.19:0/3478892246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75401b640 con 0x56512aa4ad50 2024-04-25T12:06:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc73ffff640 1 -- 172.21.15.19:0/3478892246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc75401b7e0 con 0x56512aa4ad50 2024-04-25T12:06:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc769c6a740 1 -- 172.21.15.19:0/3478892246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7500bd370 con 0x56512aa4ad50 2024-04-25T12:06:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc73ffff640 1 -- 172.21.15.19:0/3478892246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc754003e40 con 0x56512aa4ad50 2024-04-25T12:06:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc769c6a740 10 client.15135 fetch_fsmap learned FSMap version 26 2024-04-25T12:06:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc769c6a740 1 -- 172.21.15.19:0/3478892246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56512aa5e6d0 con 0x56512aa4ad50 2024-04-25T12:06:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc73ffff640 1 -- 172.21.15.19:0/3478892246 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc7500bd370 con 0x56512aa4ad50 2024-04-25T12:06:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc73ffff640 1 -- 172.21.15.19:0/3478892246 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc754019020 con 0x56512aa4ad50 2024-04-25T12:06:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc769c6a740 10 client.15135 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:06:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc769c6a740 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T12:06:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc769c6a740 20 client.15135 populate_metadata read hostname 'smithi019' 2024-04-25T12:06:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc769c6a740 1 --2- 172.21.15.19:0/3478892246 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x56512aa60d70 0x56512aa621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc769c6a740 4 client.15135 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T12:06:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.076+0000 7fc769c6a740 1 -- 172.21.15.19:0/3478892246 --> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56512a9bc670 con 0x56512aa60d70 2024-04-25T12:06:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.077+0000 7fc764ced640 1 --2- 172.21.15.19:0/3478892246 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x56512aa60d70 0x56512aa621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.077+0000 7fc73ffff640 10 client.15135 ms_handle_connect on v2:172.21.15.150:6838/2553528303 2024-04-25T12:06:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.077+0000 7fc764ced640 1 --2- 172.21.15.19:0/3478892246 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x56512aa60d70 0x56512aa621e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cbf75dc08a7dfcda server_cookie=dfcd3249b20fdd72 in_seq=0 out_seq=0 2024-04-25T12:06:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc73ffff640 1 -- 172.21.15.19:0/3478892246 <== mds.1 v2:172.21.15.150:6838/2553528303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56512a9bc670 con 0x56512aa60d70 2024-04-25T12:06:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc73ffff640 10 client.15135 handle_command_reply: tid=0 2024-04-25T12:06:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc769c6a740 1 client.15135 shutdown 2024-04-25T12:06:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc769c6a740 1 -- 172.21.15.19:0/3478892246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4ad50 msgr2=0x56512aa54250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc769c6a740 1 --2- 172.21.15.19:0/3478892246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4ad50 0x56512aa54250 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc75400bad0 tx=0x7fc75400bf90 comp rx=0 tx=0).stop 2024-04-25T12:06:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc769c6a740 1 -- 172.21.15.19:0/3478892246 shutdown_connections 2024-04-25T12:06:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc769c6a740 1 --2- 172.21.15.19:0/3478892246 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x56512aa60d70 0x56512aa621e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc769c6a740 1 --2- 172.21.15.19:0/3478892246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4b710 0x56512aa5b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc769c6a740 1 --2- 172.21.15.19:0/3478892246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4ad50 0x56512aa54250 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc73ffff640 0 client.15135 ms_handle_reset on v2:172.21.15.150:6838/2553528303 2024-04-25T12:06:35.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc769c6a740 1 --2- 172.21.15.19:0/3478892246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa49b80 0x56512aa53d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc769c6a740 1 -- 172.21.15.19:0/3478892246 >> 172.21.15.19:0/3478892246 conn(0x56512aa45070 msgr2=0x56512aa441d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:35.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc769c6a740 1 -- 172.21.15.19:0/3478892246 shutdown_connections 2024-04-25T12:06:35.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc769c6a740 1 -- 172.21.15.19:0/3478892246 wait complete. 2024-04-25T12:06:35.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.078+0000 7fc769c6a740 20 client.15135 trim_cache size 0 max 16384 2024-04-25T12:06:35.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.096+0000 7fc769c6a740 1 Processor -- start 2024-04-25T12:06:35.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.096+0000 7fc769c6a740 1 -- start start 2024-04-25T12:06:35.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.096+0000 7fc769c6a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa49b80 0x56512aa400b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.096+0000 7fc769c6a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4ad50 0x56512aa5b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.096+0000 7fc769c6a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4b710 0x56512aa4e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.096+0000 7fc769c6a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56512aa60520 con 0x56512aa4b710 2024-04-25T12:06:35.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.096+0000 7fc769c6a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56512aa60bf0 con 0x56512aa4ad50 2024-04-25T12:06:35.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.096+0000 7fc769c6a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56512aa612c0 con 0x56512aa49b80 2024-04-25T12:06:35.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.096+0000 7fc7654ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4b710 0x56512aa4e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.096+0000 7fc7654ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4b710 0x56512aa4e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36564/0 (socket says 172.21.15.19:36564) 2024-04-25T12:06:35.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.097+0000 7fc7654ee640 1 -- 172.21.15.19:0/2381504080 learned_addr learned my addr 172.21.15.19:0/2381504080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:35.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.097+0000 7fc764ced640 1 --2- 172.21.15.19:0/2381504080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa49b80 0x56512aa400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.097+0000 7fc75ffff640 1 --2- 172.21.15.19:0/2381504080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4ad50 0x56512aa5b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.097+0000 7fc7654ee640 1 -- 172.21.15.19:0/2381504080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa49b80 msgr2=0x56512aa400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.097+0000 7fc7654ee640 1 --2- 172.21.15.19:0/2381504080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa49b80 0x56512aa400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.097+0000 7fc7654ee640 1 -- 172.21.15.19:0/2381504080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4ad50 msgr2=0x56512aa5b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.097+0000 7fc7654ee640 1 --2- 172.21.15.19:0/2381504080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4ad50 0x56512aa5b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.097+0000 7fc7654ee640 1 -- 172.21.15.19:0/2381504080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56512aa526f0 con 0x56512aa4b710 2024-04-25T12:06:35.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.097+0000 7fc75ffff640 1 --2- 172.21.15.19:0/2381504080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4ad50 0x56512aa5b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:35.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.097+0000 7fc7654ee640 1 --2- 172.21.15.19:0/2381504080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4b710 0x56512aa4e890 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc758010af0 tx=0x7fc75801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da29520fb4f358e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:35.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.097+0000 7fc7467fc640 1 -- 172.21.15.19:0/2381504080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758003bf0 con 0x56512aa4b710 2024-04-25T12:06:35.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.097+0000 7fc7467fc640 1 -- 172.21.15.19:0/2381504080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc758025070 con 0x56512aa4b710 2024-04-25T12:06:35.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.097+0000 7fc7467fc640 1 -- 172.21.15.19:0/2381504080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75802a7f0 con 0x56512aa4b710 2024-04-25T12:06:35.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 -- 172.21.15.19:0/2381504080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4b710 msgr2=0x56512aa4e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 --2- 172.21.15.19:0/2381504080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4b710 0x56512aa4e890 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc758010af0 tx=0x7fc75801f000 comp rx=0 tx=0).stop 2024-04-25T12:06:35.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 -- 172.21.15.19:0/2381504080 shutdown_connections 2024-04-25T12:06:35.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 --2- 172.21.15.19:0/2381504080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa4b710 0x56512aa4e890 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 --2- 172.21.15.19:0/2381504080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4ad50 0x56512aa5b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 --2- 172.21.15.19:0/2381504080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa49b80 0x56512aa400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 -- 172.21.15.19:0/2381504080 >> 172.21.15.19:0/2381504080 conn(0x56512aa45070 msgr2=0x56512aa45450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:35.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 -- 172.21.15.19:0/2381504080 shutdown_connections 2024-04-25T12:06:35.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 -- 172.21.15.19:0/2381504080 wait complete. 2024-04-25T12:06:35.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 -- start start 2024-04-25T12:06:35.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 Processor -- start 2024-04-25T12:06:35.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa49b80 0x56512aa50650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4ad50 0x56512aa50b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa4b710 0x56512aac4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56512aa62060 con 0x56512aa49b80 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56512aa61990 con 0x56512aa4ad50 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x56512aa62730 con 0x56512aa4b710 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.098+0000 7fc769c6a740 10 client.0 _ll_register_callbacks cb 0x7fc754004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc764ced640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa49b80 0x56512aa50650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc764ced640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa49b80 0x56512aa50650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36580/0 (socket says 172.21.15.19:36580) 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc764ced640 1 -- 172.21.15.19:0/276313116 learned_addr learned my addr 172.21.15.19:0/276313116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc75ffff640 1 --2- 172.21.15.19:0/276313116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4ad50 0x56512aa50b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc7654ee640 1 --2- 172.21.15.19:0/276313116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa4b710 0x56512aac4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc764ced640 1 -- 172.21.15.19:0/276313116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa4b710 msgr2=0x56512aac4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc764ced640 1 --2- 172.21.15.19:0/276313116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa4b710 0x56512aac4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc764ced640 1 -- 172.21.15.19:0/276313116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4ad50 msgr2=0x56512aa50b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc764ced640 1 --2- 172.21.15.19:0/276313116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4ad50 0x56512aa50b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc764ced640 1 -- 172.21.15.19:0/276313116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56512aac5250 con 0x56512aa49b80 2024-04-25T12:06:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc764ced640 1 --2- 172.21.15.19:0/276313116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa49b80 0x56512aa50650 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc7500ee800 tx=0x7fc75003b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56dda6ee087f6a7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc73ffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:06:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc73ffff640 1 -- 172.21.15.19:0/276313116 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750049040 con 0x56512aa49b80 2024-04-25T12:06:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc73ffff640 1 -- 172.21.15.19:0/276313116 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc7500baf70 con 0x56512aa49b80 2024-04-25T12:06:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc769c6a740 1 -- 172.21.15.19:0/276313116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc758025070 con 0x56512aa49b80 2024-04-25T12:06:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc73ffff640 1 -- 172.21.15.19:0/276313116 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc7500f0e60 con 0x56512aa49b80 2024-04-25T12:06:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc769c6a740 10 client.15141 fetch_fsmap learned FSMap version 26 2024-04-25T12:06:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.099+0000 7fc769c6a740 1 -- 172.21.15.19:0/276313116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56512aac63f0 con 0x56512aa49b80 2024-04-25T12:06:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.100+0000 7fc73ffff640 1 -- 172.21.15.19:0/276313116 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc758025070 con 0x56512aa49b80 2024-04-25T12:06:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.100+0000 7fc73ffff640 1 -- 172.21.15.19:0/276313116 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc7500b92e0 con 0x56512aa49b80 2024-04-25T12:06:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.100+0000 7fc769c6a740 10 client.15141 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:06:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.100+0000 7fc769c6a740 10 client.15141 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T12:06:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.100+0000 7fc769c6a740 20 client.15141 populate_metadata read hostname 'smithi019' 2024-04-25T12:06:35.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.100+0000 7fc769c6a740 1 --2- 172.21.15.19:0/276313116 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x56512aacacb0 0x56512aac99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.100+0000 7fc769c6a740 4 client.15141 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.100+0000 7fc769c6a740 1 -- 172.21.15.19:0/276313116 --> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] -- command(tid 0: {"prefix": "status"}) -- 0x56512a9b64e0 con 0x56512aacacb0 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.100+0000 7fc75ffff640 1 --2- 172.21.15.19:0/276313116 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x56512aacacb0 0x56512aac99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.100+0000 7fc73ffff640 10 client.15141 ms_handle_connect on v2:172.21.15.150:6838/2553528303 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.100+0000 7fc75ffff640 1 --2- 172.21.15.19:0/276313116 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x56512aacacb0 0x56512aac99c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe64b2029036d4f3 server_cookie=1250c4481fc9e940 in_seq=0 out_seq=0 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc73ffff640 1 -- 172.21.15.19:0/276313116 <== mds.1 v2:172.21.15.150:6838/2553528303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56512a9b64e0 con 0x56512aacacb0 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc73ffff640 10 client.15141 handle_command_reply: tid=0 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc769c6a740 1 client.15141 shutdown 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc769c6a740 1 -- 172.21.15.19:0/276313116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa49b80 msgr2=0x56512aa50650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc769c6a740 1 --2- 172.21.15.19:0/276313116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa49b80 0x56512aa50650 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc7500ee800 tx=0x7fc75003b460 comp rx=0 tx=0).stop 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc769c6a740 1 -- 172.21.15.19:0/276313116 shutdown_connections 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc769c6a740 1 --2- 172.21.15.19:0/276313116 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x56512aacacb0 0x56512aac99c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc769c6a740 1 --2- 172.21.15.19:0/276313116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56512aa4b710 0x56512aac4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc73ffff640 0 client.15141 ms_handle_reset on v2:172.21.15.150:6838/2553528303 2024-04-25T12:06:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc769c6a740 1 --2- 172.21.15.19:0/276313116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56512aa4ad50 0x56512aa50b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc769c6a740 1 --2- 172.21.15.19:0/276313116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56512aa49b80 0x56512aa50650 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc769c6a740 1 -- 172.21.15.19:0/276313116 >> 172.21.15.19:0/276313116 conn(0x56512aa45070 msgr2=0x56512aa400b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:35.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc769c6a740 1 -- 172.21.15.19:0/276313116 shutdown_connections 2024-04-25T12:06:35.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc769c6a740 1 -- 172.21.15.19:0/276313116 wait complete. 2024-04-25T12:06:35.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.101+0000 7fc769c6a740 20 client.15141 trim_cache size 0 max 16384 2024-04-25T12:06:35.106 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:06:35.106 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "be62e290-02fa-11ef-bc93-c7b262605968", 2024-04-25T12:06:35.106 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-04-25T12:06:35.106 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24500, 2024-04-25T12:06:35.106 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T12:06:35.107 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T12:06:35.107 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T12:06:35.107 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 87.406385139999998, 2024-04-25T12:06:35.107 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-25T12:06:35.107 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 87, 2024-04-25T12:06:35.107 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T12:06:35.107 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 97.171271724999997 2024-04-25T12:06:35.107 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:06:35.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.102+0000 7fc766f51640 1 -- 172.21.15.19:0/3749477599 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc7480a3c30 msgr2=0x7fc7480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.102+0000 7fc766f51640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc7480a3c30 0x7fc7480a60f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc7580098b0 tx=0x7fc758009d90 comp rx=0 tx=0).stop 2024-04-25T12:06:35.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.102+0000 7fc766f51640 1 -- 172.21.15.19:0/3749477599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc760102a30 msgr2=0x7fc7601023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.102+0000 7fc766f51640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc760102a30 0x7fc7601023a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc750005940 tx=0x7fc7500043b0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.102+0000 7fc766f51640 1 -- 172.21.15.19:0/3749477599 shutdown_connections 2024-04-25T12:06:35.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.102+0000 7fc766f51640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fc7480a3c30 0x7fc7480a60f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.102+0000 7fc766f51640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc760107c90 0x7fc760100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.102+0000 7fc766f51640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc760105360 0x7fc7601009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.102+0000 7fc766f51640 1 --2- 172.21.15.19:0/3749477599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc760102a30 0x7fc7601023a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.102+0000 7fc766f51640 1 -- 172.21.15.19:0/3749477599 >> 172.21.15.19:0/3749477599 conn(0x7fc7600fc440 msgr2=0x7fc7601034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:35.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.102+0000 7fc766f51640 1 -- 172.21.15.19:0/3749477599 shutdown_connections 2024-04-25T12:06:35.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.102+0000 7fc766f51640 1 -- 172.21.15.19:0/3749477599 wait complete. 2024-04-25T12:06:35.111 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T12:06:35.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:34 smithi169 ceph-mon[32710]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T12:06:35.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:34 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2263735681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:06:35.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:34 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/601635986' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:06:35.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:34 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3623065334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:06:35.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:34 smithi150 ceph-mon[32250]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T12:06:35.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:34 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2263735681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:06:35.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:34 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/601635986' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:06:35.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:34 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3623065334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:06:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.218+0000 7f3d0e08a640 1 -- 172.21.15.19:0/3595647218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d08101fe0 msgr2=0x7f3d08102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.218+0000 7f3d0e08a640 1 --2- 172.21.15.19:0/3595647218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d08101fe0 0x7f3d08102460 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009900 tx=0x7f3cf002a410 comp rx=0 tx=0).stop 2024-04-25T12:06:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.218+0000 7f3d0e08a640 1 -- 172.21.15.19:0/3595647218 shutdown_connections 2024-04-25T12:06:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.218+0000 7f3d0e08a640 1 --2- 172.21.15.19:0/3595647218 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d081029a0 0x7f3d081092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.218+0000 7f3d0e08a640 1 --2- 172.21.15.19:0/3595647218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d08101fe0 0x7f3d08102460 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.218+0000 7f3d0e08a640 1 --2- 172.21.15.19:0/3595647218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08100e90 0x7f3d081012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.218+0000 7f3d0e08a640 1 -- 172.21.15.19:0/3595647218 >> 172.21.15.19:0/3595647218 conn(0x7f3d080fc400 msgr2=0x7f3d080fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.218+0000 7f3d0e08a640 1 -- 172.21.15.19:0/3595647218 shutdown_connections 2024-04-25T12:06:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.218+0000 7f3d0e08a640 1 -- 172.21.15.19:0/3595647218 wait complete. 2024-04-25T12:06:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.218+0000 7f3d0e08a640 1 Processor -- start 2024-04-25T12:06:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.218+0000 7f3d0e08a640 1 -- start start 2024-04-25T12:06:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d0e08a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08100e90 0x7f3d08069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d0e08a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d08101fe0 0x7f3d0819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d0e08a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d081029a0 0x7f3d0819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d0e08a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0810cb20 con 0x7f3d081029a0 2024-04-25T12:06:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d0e08a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0810c9a0 con 0x7f3d08100e90 2024-04-25T12:06:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d0e08a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0810cca0 con 0x7f3d08101fe0 2024-04-25T12:06:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d07fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d081029a0 0x7f3d0819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d07fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d081029a0 0x7f3d0819f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36612/0 (socket says 172.21.15.19:36612) 2024-04-25T12:06:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d07fff640 1 -- 172.21.15.19:0/215847150 learned_addr learned my addr 172.21.15.19:0/215847150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d077fe640 1 --2- 172.21.15.19:0/215847150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08100e90 0x7f3d08069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3cfeffd640 1 --2- 172.21.15.19:0/215847150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d08101fe0 0x7f3d0819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d07fff640 1 -- 172.21.15.19:0/215847150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d08101fe0 msgr2=0x7f3d0819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d07fff640 1 --2- 172.21.15.19:0/215847150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d08101fe0 0x7f3d0819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d07fff640 1 -- 172.21.15.19:0/215847150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08100e90 msgr2=0x7f3d08069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d07fff640 1 --2- 172.21.15.19:0/215847150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08100e90 0x7f3d08069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d07fff640 1 -- 172.21.15.19:0/215847150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0819feb0 con 0x7f3d081029a0 2024-04-25T12:06:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d07fff640 1 --2- 172.21.15.19:0/215847150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d081029a0 0x7f3d0819f5f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000eaa0 tx=0x7f3d0000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d057fa640 1 -- 172.21.15.19:0/215847150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0000cdb0 con 0x7f3d081029a0 2024-04-25T12:06:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d057fa640 1 -- 172.21.15.19:0/215847150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3d00005020 con 0x7f3d081029a0 2024-04-25T12:06:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d0e08a640 1 -- 172.21.15.19:0/215847150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d081a01a0 con 0x7f3d081029a0 2024-04-25T12:06:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d057fa640 1 -- 172.21.15.19:0/215847150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d000116d0 con 0x7f3d081029a0 2024-04-25T12:06:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.219+0000 7f3d0e08a640 1 -- 172.21.15.19:0/215847150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d081a06c0 con 0x7f3d081029a0 2024-04-25T12:06:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3d057fa640 1 -- 172.21.15.19:0/215847150 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3d000051d0 con 0x7f3d081029a0 2024-04-25T12:06:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3d10da2740 1 Processor -- start 2024-04-25T12:06:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3d10da2740 1 -- start start 2024-04-25T12:06:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3d10da2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 0x557d92fc73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3d10da2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8160 0x557d92fc85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3d10da2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8b20 0x557d92fcf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3d10da2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d92f27c50 con 0x557d92fc8b20 2024-04-25T12:06:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3d10da2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557d92ec4830 con 0x557d92fc8160 2024-04-25T12:06:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3d10da2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557d92f23c10 con 0x557d92fc6f90 2024-04-25T12:06:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3cfeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8b20 0x557d92fcf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3cfeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8b20 0x557d92fcf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36616/0 (socket says 172.21.15.19:36616) 2024-04-25T12:06:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3cfeffd640 1 -- 172.21.15.19:0/1070907104 learned_addr learned my addr 172.21.15.19:0/1070907104 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3d077fe640 1 --2- 172.21.15.19:0/1070907104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8160 0x557d92fc85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3cfeffd640 1 --2- 172.21.15.19:0/1070907104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 0x557d92fc73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.222+0000 7f3d057fa640 1 --2- 172.21.15.19:0/215847150 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3cf40a3bb0 0x7f3cf40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.223+0000 7f3d077fe640 1 -- 172.21.15.19:0/1070907104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 msgr2=0x557d92fc73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.223+0000 7f3d077fe640 1 --2- 172.21.15.19:0/1070907104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 0x557d92fc73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.223+0000 7f3d077fe640 1 -- 172.21.15.19:0/1070907104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8b20 msgr2=0x557d92fcf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.223+0000 7f3d077fe640 1 --2- 172.21.15.19:0/1070907104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8b20 0x557d92fcf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.223+0000 7f3d07fff640 1 --2- 172.21.15.19:0/215847150 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3cf40a3bb0 0x7f3cf40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.223+0000 7f3d077fe640 1 -- 172.21.15.19:0/1070907104 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d92c3a2d0 con 0x557d92fc8160 2024-04-25T12:06:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.223+0000 7f3d057fa640 1 -- 172.21.15.19:0/215847150 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f3d000d0610 con 0x7f3d081029a0 2024-04-25T12:06:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.223+0000 7f3d077fe640 1 --2- 172.21.15.19:0/1070907104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8160 0x557d92fc85e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009c20 tx=0x7f3cf8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e764ff9f921c5d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.223+0000 7f3d07fff640 1 --2- 172.21.15.19:0/215847150 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3cf40a3bb0 0x7f3cf40a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3d000992a0 tx=0x7f3d00066fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.223+0000 7f3cfdffb640 1 -- 172.21.15.19:0/1070907104 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf801e070 con 0x557d92fc8160 2024-04-25T12:06:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.223+0000 7f3cfdffb640 1 -- 172.21.15.19:0/1070907104 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3cf8019900 con 0x557d92fc8160 2024-04-25T12:06:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.223+0000 7f3cfdffb640 1 -- 172.21.15.19:0/1070907104 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800ea30 con 0x557d92fc8160 2024-04-25T12:06:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.224+0000 7f3d10da2740 1 -- 172.21.15.19:0/1070907104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8160 msgr2=0x557d92fc85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.224+0000 7f3d10da2740 1 --2- 172.21.15.19:0/1070907104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8160 0x557d92fc85e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009c20 tx=0x7f3cf8011810 comp rx=0 tx=0).stop 2024-04-25T12:06:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.224+0000 7f3d10da2740 1 -- 172.21.15.19:0/1070907104 shutdown_connections 2024-04-25T12:06:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.224+0000 7f3d10da2740 1 --2- 172.21.15.19:0/1070907104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8b20 0x557d92fcf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.224+0000 7f3d10da2740 1 --2- 172.21.15.19:0/1070907104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8160 0x557d92fc85e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.224+0000 7f3d10da2740 1 --2- 172.21.15.19:0/1070907104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 0x557d92fc73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.224+0000 7f3d10da2740 1 -- 172.21.15.19:0/1070907104 >> 172.21.15.19:0/1070907104 conn(0x557d92fbe070 msgr2=0x557d92fbe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.224+0000 7f3d10da2740 1 -- 172.21.15.19:0/1070907104 shutdown_connections 2024-04-25T12:06:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.224+0000 7f3d10da2740 1 -- 172.21.15.19:0/1070907104 wait complete. 2024-04-25T12:06:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.224+0000 7f3d10da2740 1 -- start start 2024-04-25T12:06:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.224+0000 7f3d10da2740 1 Processor -- start 2024-04-25T12:06:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d10da2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 0x557d92fd1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d10da2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8160 0x557d92fd1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d10da2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8b20 0x557d92fd89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d10da2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d92fcfb30 con 0x557d92fc8160 2024-04-25T12:06:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d10da2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557d92ec2470 con 0x557d92fc8b20 2024-04-25T12:06:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d10da2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557d92fcfcb0 con 0x557d92fc6f90 2024-04-25T12:06:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d10da2740 10 client.0 _ll_register_callbacks cb 0x557d92ee7f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:06:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8160 0x557d92fd1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8160 0x557d92fd1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36618/0 (socket says 172.21.15.19:36618) 2024-04-25T12:06:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d077fe640 1 -- 172.21.15.19:0/1820334521 learned_addr learned my addr 172.21.15.19:0/1820334521 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3cfeffd640 1 --2- 172.21.15.19:0/1820334521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8b20 0x557d92fd89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3cfeffd640 1 --2- 172.21.15.19:0/1820334521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 0x557d92fd1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d077fe640 1 -- 172.21.15.19:0/1820334521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 msgr2=0x557d92fd1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d077fe640 1 --2- 172.21.15.19:0/1820334521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 0x557d92fd1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d077fe640 1 -- 172.21.15.19:0/1820334521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8b20 msgr2=0x557d92fd89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d077fe640 1 --2- 172.21.15.19:0/1820334521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8b20 0x557d92fd89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d077fe640 1 -- 172.21.15.19:0/1820334521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d92d046b0 con 0x557d92fc8160 2024-04-25T12:06:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3cfeffd640 1 --2- 172.21.15.19:0/1820334521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 0x557d92fd1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d077fe640 1 --2- 172.21.15.19:0/1820334521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8160 0x557d92fd1960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80097b0 tx=0x7f3cf8019ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0ee30ca47ebf39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3ce6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:06:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3ce6ffd640 1 -- 172.21.15.19:0/1820334521 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf801e030 con 0x557d92fc8160 2024-04-25T12:06:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3ce6ffd640 1 -- 172.21.15.19:0/1820334521 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f3cf8004250 con 0x557d92fc8160 2024-04-25T12:06:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3d10da2740 1 -- 172.21.15.19:0/1820334521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cf8019900 con 0x557d92fc8160 2024-04-25T12:06:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.225+0000 7f3ce6ffd640 1 -- 172.21.15.19:0/1820334521 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3cf800e3b0 con 0x557d92fc8160 2024-04-25T12:06:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.226+0000 7f3d10da2740 10 client.15153 fetch_fsmap learned FSMap version 26 2024-04-25T12:06:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.226+0000 7f3d10da2740 1 -- 172.21.15.19:0/1820334521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557d92fdbd10 con 0x557d92fc8160 2024-04-25T12:06:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.226+0000 7f3ce6ffd640 1 -- 172.21.15.19:0/1820334521 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3cf8019900 con 0x557d92fc8160 2024-04-25T12:06:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.226+0000 7f3ce6ffd640 1 -- 172.21.15.19:0/1820334521 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3cf800ea80 con 0x557d92fc8160 2024-04-25T12:06:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.226+0000 7f3d10da2740 10 client.15153 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:06:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.226+0000 7f3d10da2740 10 client.15153 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T12:06:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.226+0000 7f3d10da2740 20 client.15153 populate_metadata read hostname 'smithi019' 2024-04-25T12:06:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.226+0000 7f3d10da2740 1 --2- 172.21.15.19:0/1820334521 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557d92fde150 0x557d92fdf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.226+0000 7f3d10da2740 4 client.15153 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T12:06:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.226+0000 7f3d10da2740 1 -- 172.21.15.19:0/1820334521 --> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d92f35670 con 0x557d92fde150 2024-04-25T12:06:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.226+0000 7f3cfeffd640 1 --2- 172.21.15.19:0/1820334521 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557d92fde150 0x557d92fdf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.227+0000 7f3ce6ffd640 10 client.15153 ms_handle_connect on v2:172.21.15.169:6838/1723439500 2024-04-25T12:06:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.227+0000 7f3cfeffd640 1 --2- 172.21.15.19:0/1820334521 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557d92fde150 0x557d92fdf5c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=287be9089c1162c8 server_cookie=3f958aade731e4d1 in_seq=0 out_seq=0 2024-04-25T12:06:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3ce6ffd640 1 -- 172.21.15.19:0/1820334521 <== mds.0 v2:172.21.15.169:6838/1723439500 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d92f35670 con 0x557d92fde150 2024-04-25T12:06:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3ce6ffd640 10 client.15153 handle_command_reply: tid=0 2024-04-25T12:06:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3d10da2740 1 client.15153 shutdown 2024-04-25T12:06:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3d10da2740 1 -- 172.21.15.19:0/1820334521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8160 msgr2=0x557d92fd1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3d10da2740 1 --2- 172.21.15.19:0/1820334521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8160 0x557d92fd1960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80097b0 tx=0x7f3cf8019ce0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3d10da2740 1 -- 172.21.15.19:0/1820334521 shutdown_connections 2024-04-25T12:06:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3d10da2740 1 --2- 172.21.15.19:0/1820334521 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557d92fde150 0x557d92fdf5c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3d10da2740 1 --2- 172.21.15.19:0/1820334521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8b20 0x557d92fd89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3ce6ffd640 0 client.15153 ms_handle_reset on v2:172.21.15.169:6838/1723439500 2024-04-25T12:06:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3d10da2740 1 --2- 172.21.15.19:0/1820334521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8160 0x557d92fd1960 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3d10da2740 1 --2- 172.21.15.19:0/1820334521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 0x557d92fd1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3d10da2740 1 -- 172.21.15.19:0/1820334521 >> 172.21.15.19:0/1820334521 conn(0x557d92fbe070 msgr2=0x557d92fbd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3d10da2740 1 -- 172.21.15.19:0/1820334521 shutdown_connections 2024-04-25T12:06:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3d10da2740 1 -- 172.21.15.19:0/1820334521 wait complete. 2024-04-25T12:06:35.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.228+0000 7f3d10da2740 20 client.15153 trim_cache size 0 max 16384 2024-04-25T12:06:35.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.246+0000 7f3d10da2740 1 Processor -- start 2024-04-25T12:06:35.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.246+0000 7f3d10da2740 1 -- start start 2024-04-25T12:06:35.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.246+0000 7f3d10da2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 0x557d92fdd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.246+0000 7f3d10da2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8160 0x557d92fb90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.246+0000 7f3d10da2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8b20 0x557d92fd0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.246+0000 7f3d10da2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d92fdd900 con 0x557d92fc8160 2024-04-25T12:06:35.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.246+0000 7f3d10da2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557d92fddfd0 con 0x557d92fc8b20 2024-04-25T12:06:35.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.246+0000 7f3d10da2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557d92fde6a0 con 0x557d92fc6f90 2024-04-25T12:06:35.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.246+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8160 0x557d92fb90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.247+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8160 0x557d92fb90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36624/0 (socket says 172.21.15.19:36624) 2024-04-25T12:06:35.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.247+0000 7f3d077fe640 1 -- 172.21.15.19:0/1475092037 learned_addr learned my addr 172.21.15.19:0/1475092037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:35.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.247+0000 7f3cfeffd640 1 --2- 172.21.15.19:0/1475092037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8b20 0x557d92fd0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.247+0000 7f3d077fe640 1 -- 172.21.15.19:0/1475092037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 msgr2=0x557d92fdd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.247+0000 7f3d077fe640 1 --2- 172.21.15.19:0/1475092037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 0x557d92fdd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.247+0000 7f3d077fe640 1 -- 172.21.15.19:0/1475092037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8b20 msgr2=0x557d92fd0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.247+0000 7f3d077fe640 1 --2- 172.21.15.19:0/1475092037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8b20 0x557d92fd0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.247+0000 7f3d077fe640 1 -- 172.21.15.19:0/1475092037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d92fdf8f0 con 0x557d92fc8160 2024-04-25T12:06:35.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.247+0000 7f3cfeffd640 1 --2- 172.21.15.19:0/1475092037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8b20 0x557d92fd0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:06:35.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.247+0000 7f3d077fe640 1 --2- 172.21.15.19:0/1475092037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8160 0x557d92fb90b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009b40 tx=0x7f3cf8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3742766cffa144 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:35.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.247+0000 7f3cfd7fa640 1 -- 172.21.15.19:0/1475092037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf801e070 con 0x557d92fc8160 2024-04-25T12:06:35.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.247+0000 7f3cfd7fa640 1 -- 172.21.15.19:0/1475092037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3cf8028070 con 0x557d92fc8160 2024-04-25T12:06:35.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.247+0000 7f3cfd7fa640 1 -- 172.21.15.19:0/1475092037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800eca0 con 0x557d92fc8160 2024-04-25T12:06:35.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 -- 172.21.15.19:0/1475092037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8160 msgr2=0x557d92fb90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 --2- 172.21.15.19:0/1475092037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8160 0x557d92fb90b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009b40 tx=0x7f3cf8000bc0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 -- 172.21.15.19:0/1475092037 shutdown_connections 2024-04-25T12:06:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 --2- 172.21.15.19:0/1475092037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc8b20 0x557d92fd0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 --2- 172.21.15.19:0/1475092037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8160 0x557d92fb90b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 --2- 172.21.15.19:0/1475092037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc6f90 0x557d92fdd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 -- 172.21.15.19:0/1475092037 >> 172.21.15.19:0/1475092037 conn(0x557d92fbe070 msgr2=0x557d92fbe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 -- 172.21.15.19:0/1475092037 shutdown_connections 2024-04-25T12:06:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 -- 172.21.15.19:0/1475092037 wait complete. 2024-04-25T12:06:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 -- start start 2024-04-25T12:06:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 Processor -- start 2024-04-25T12:06:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc6f90 0x557d92fd3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc8160 0x557d92fd3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8b20 0x557d92fd6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557d92fdf440 con 0x557d92fc8b20 2024-04-25T12:06:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557d92fded70 con 0x557d92fc6f90 2024-04-25T12:06:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.248+0000 7f3d10da2740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557d92fdfb10 con 0x557d92fc8160 2024-04-25T12:06:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3d10da2740 10 client.0 _ll_register_callbacks cb 0x7f3cf801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:06:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3cfeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8b20 0x557d92fd6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3cfeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8b20 0x557d92fd6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36636/0 (socket says 172.21.15.19:36636) 2024-04-25T12:06:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3cfeffd640 1 -- 172.21.15.19:0/802089936 learned_addr learned my addr 172.21.15.19:0/802089936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3d077fe640 1 --2- 172.21.15.19:0/802089936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc8160 0x557d92fd3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3cfeffd640 1 --2- 172.21.15.19:0/802089936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc6f90 0x557d92fd3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3cfeffd640 1 -- 172.21.15.19:0/802089936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc8160 msgr2=0x557d92fd3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3cfeffd640 1 --2- 172.21.15.19:0/802089936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc8160 0x557d92fd3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3cfeffd640 1 -- 172.21.15.19:0/802089936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc6f90 msgr2=0x557d92fd3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:06:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3cfeffd640 1 --2- 172.21.15.19:0/802089936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc6f90 0x557d92fd3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3cfeffd640 1 -- 172.21.15.19:0/802089936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d92fd7390 con 0x557d92fc8b20 2024-04-25T12:06:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3d077fe640 1 --2- 172.21.15.19:0/802089936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc8160 0x557d92fd3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:35.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3cfeffd640 1 --2- 172.21.15.19:0/802089936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8b20 0x557d92fd6da0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0032fd0 tx=0x7f3cf0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e566c641e743a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3ce6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:06:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3ce6ffd640 1 -- 172.21.15.19:0/802089936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0035040 con 0x557d92fc8b20 2024-04-25T12:06:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3ce6ffd640 1 -- 172.21.15.19:0/802089936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f3cf0031890 con 0x557d92fc8b20 2024-04-25T12:06:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3d10da2740 1 -- 172.21.15.19:0/802089936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cf8028070 con 0x557d92fc8b20 2024-04-25T12:06:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3ce6ffd640 1 -- 172.21.15.19:0/802089936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3cf0031a90 con 0x557d92fc8b20 2024-04-25T12:06:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3d10da2740 10 client.15159 fetch_fsmap learned FSMap version 26 2024-04-25T12:06:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3d10da2740 1 -- 172.21.15.19:0/802089936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557d93044330 con 0x557d92fc8b20 2024-04-25T12:06:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.249+0000 7f3ce6ffd640 1 -- 172.21.15.19:0/802089936 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3cf8028070 con 0x557d92fc8b20 2024-04-25T12:06:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.250+0000 7f3ce6ffd640 1 -- 172.21.15.19:0/802089936 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3cf003a040 con 0x557d92fc8b20 2024-04-25T12:06:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.250+0000 7f3d10da2740 10 client.15159 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:06:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.250+0000 7f3d10da2740 10 client.15159 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T12:06:35.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.250+0000 7f3d10da2740 20 client.15159 populate_metadata read hostname 'smithi019' 2024-04-25T12:06:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.250+0000 7f3d10da2740 1 --2- 172.21.15.19:0/802089936 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557d93048f20 0x557d93045a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.250+0000 7f3d10da2740 4 client.15159 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T12:06:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.250+0000 7f3d10da2740 1 -- 172.21.15.19:0/802089936 --> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] -- command(tid 0: {"prefix": "status"}) -- 0x557d92f2f4e0 con 0x557d93048f20 2024-04-25T12:06:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.250+0000 7f3d077fe640 1 --2- 172.21.15.19:0/802089936 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557d93048f20 0x557d93045a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.250+0000 7f3d077fe640 1 --2- 172.21.15.19:0/802089936 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557d93048f20 0x557d93045a30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e760fc3386b6b3 server_cookie=921487febb892e19 in_seq=0 out_seq=0 2024-04-25T12:06:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.250+0000 7f3ce6ffd640 10 client.15159 ms_handle_connect on v2:172.21.15.169:6838/1723439500 2024-04-25T12:06:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3ce6ffd640 1 -- 172.21.15.19:0/802089936 <== mds.0 v2:172.21.15.169:6838/1723439500 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557d92f2f4e0 con 0x557d93048f20 2024-04-25T12:06:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3ce6ffd640 10 client.15159 handle_command_reply: tid=0 2024-04-25T12:06:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d10da2740 1 client.15159 shutdown 2024-04-25T12:06:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d10da2740 1 -- 172.21.15.19:0/802089936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8b20 msgr2=0x557d92fd6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d10da2740 1 --2- 172.21.15.19:0/802089936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8b20 0x557d92fd6da0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0032fd0 tx=0x7f3cf0009260 comp rx=0 tx=0).stop 2024-04-25T12:06:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d10da2740 1 -- 172.21.15.19:0/802089936 shutdown_connections 2024-04-25T12:06:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d10da2740 1 --2- 172.21.15.19:0/802089936 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557d93048f20 0x557d93045a30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d10da2740 1 --2- 172.21.15.19:0/802089936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557d92fc8b20 0x557d92fd6da0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d10da2740 1 --2- 172.21.15.19:0/802089936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557d92fc8160 0x557d92fd3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3ce6ffd640 0 client.15159 ms_handle_reset on v2:172.21.15.169:6838/1723439500 2024-04-25T12:06:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d10da2740 1 --2- 172.21.15.19:0/802089936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557d92fc6f90 0x557d92fd3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d10da2740 1 -- 172.21.15.19:0/802089936 >> 172.21.15.19:0/802089936 conn(0x557d92fbe070 msgr2=0x557d92fdd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d10da2740 1 -- 172.21.15.19:0/802089936 shutdown_connections 2024-04-25T12:06:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d10da2740 1 -- 172.21.15.19:0/802089936 wait complete. 2024-04-25T12:06:35.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d10da2740 20 client.15159 trim_cache size 0 max 16384 2024-04-25T12:06:35.256 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:06:35.256 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "be62e290-02fa-11ef-bc93-c7b262605968", 2024-04-25T12:06:35.257 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T12:06:35.257 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24514, 2024-04-25T12:06:35.257 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T12:06:35.257 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T12:06:35.257 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T12:06:35.257 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 90.575216369000003, 2024-04-25T12:06:35.257 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-25T12:06:35.257 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 87, 2024-04-25T12:06:35.257 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T12:06:35.257 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 106.111187805 2024-04-25T12:06:35.257 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:06:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d0e08a640 1 -- 172.21.15.19:0/215847150 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3cf40a3bb0 msgr2=0x7f3cf40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d0e08a640 1 --2- 172.21.15.19:0/215847150 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3cf40a3bb0 0x7f3cf40a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3d000992a0 tx=0x7f3d00066fd0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d0e08a640 1 -- 172.21.15.19:0/215847150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d081029a0 msgr2=0x7f3d0819f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:35.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.251+0000 7f3d0e08a640 1 --2- 172.21.15.19:0/215847150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d081029a0 0x7f3d0819f5f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000eaa0 tx=0x7f3d0000ef60 comp rx=0 tx=0).stop 2024-04-25T12:06:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.252+0000 7f3d0e08a640 1 -- 172.21.15.19:0/215847150 shutdown_connections 2024-04-25T12:06:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.252+0000 7f3d0e08a640 1 --2- 172.21.15.19:0/215847150 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3cf40a3bb0 0x7f3cf40a6070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.252+0000 7f3d0e08a640 1 --2- 172.21.15.19:0/215847150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d081029a0 0x7f3d0819f5f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.252+0000 7f3d0e08a640 1 --2- 172.21.15.19:0/215847150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3d08101fe0 0x7f3d0819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.252+0000 7f3d0e08a640 1 --2- 172.21.15.19:0/215847150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d08100e90 0x7f3d08069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.252+0000 7f3d0e08a640 1 -- 172.21.15.19:0/215847150 >> 172.21.15.19:0/215847150 conn(0x7f3d080fc400 msgr2=0x7f3d08105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.252+0000 7f3d0e08a640 1 -- 172.21.15.19:0/215847150 shutdown_connections 2024-04-25T12:06:35.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:35.252+0000 7f3d0e08a640 1 -- 172.21.15.19:0/215847150 wait complete. 2024-04-25T12:06:35.261 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T12:06:35.261 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T12:06:35.271 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T12:06:35.271 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T12:06:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:06:36.939 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:36 smithi019 ceph-mon[29213]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T12:06:36.957 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:36 smithi169 ceph-mon[32710]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T12:06:37.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:36 smithi150 ceph-mon[32250]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-25T12:06:37.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.285+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/1896859713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ebc104640 msgr2=0x7f9ebc10aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:37.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.285+0000 7f9ec3cb2700 1 --2- 172.21.15.19:0/1896859713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ebc104640 0x7f9ebc10aef0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400b0d0 tx=0x7f9eb400b3e0 comp rx=0 tx=0).stop 2024-04-25T12:06:37.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.285+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/1896859713 shutdown_connections 2024-04-25T12:06:37.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.285+0000 7f9ec3cb2700 1 --2- 172.21.15.19:0/1896859713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ebc104640 0x7f9ebc10aef0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:37.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.285+0000 7f9ec3cb2700 1 --2- 172.21.15.19:0/1896859713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ebc102aa0 0x7f9ebc102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.285+0000 7f9ec3cb2700 1 --2- 172.21.15.19:0/1896859713 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ebc103ca0 0x7f9ebc104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.285+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/1896859713 >> 172.21.15.19:0/1896859713 conn(0x7f9ebc0fe040 msgr2=0x7f9ebc100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.286+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/1896859713 shutdown_connections 2024-04-25T12:06:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.286+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/1896859713 wait complete. 2024-04-25T12:06:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.286+0000 7f9ec3cb2700 1 Processor -- start 2024-04-25T12:06:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.286+0000 7f9ec3cb2700 1 -- start start 2024-04-25T12:06:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.286+0000 7f9ec3cb2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ebc102aa0 0x7f9ebc0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.286+0000 7f9ec3cb2700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ebc103ca0 0x7f9ebc074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.286+0000 7f9ec3cb2700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ebc104640 0x7f9ebc075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.286+0000 7f9ec3cb2700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc10e680 con 0x7f9ebc104640 2024-04-25T12:06:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.286+0000 7f9ec3cb2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc10e500 con 0x7f9ebc102aa0 2024-04-25T12:06:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.286+0000 7f9ec3cb2700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc10e800 con 0x7f9ebc103ca0 2024-04-25T12:06:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.286+0000 7f9ec224f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ebc104640 0x7f9ebc075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:37.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.286+0000 7f9ec224f700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ebc104640 0x7f9ebc075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36652/0 (socket says 172.21.15.19:36652) 2024-04-25T12:06:37.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.286+0000 7f9ec224f700 1 -- 172.21.15.19:0/2815149217 learned_addr learned my addr 172.21.15.19:0/2815149217 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:37.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.287+0000 7f9ec124d700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ebc103ca0 0x7f9ebc074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:37.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.287+0000 7f9ec1a4e700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ebc102aa0 0x7f9ebc0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:37.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.287+0000 7f9ec224f700 1 -- 172.21.15.19:0/2815149217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ebc103ca0 msgr2=0x7f9ebc074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:37.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.287+0000 7f9ec224f700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ebc103ca0 0x7f9ebc074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:37.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.287+0000 7f9ec224f700 1 -- 172.21.15.19:0/2815149217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ebc102aa0 msgr2=0x7f9ebc0767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:37.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.287+0000 7f9ec224f700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ebc102aa0 0x7f9ebc0767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:37.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.287+0000 7f9ec224f700 1 -- 172.21.15.19:0/2815149217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ebc075c20 con 0x7f9ebc104640 2024-04-25T12:06:37.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.287+0000 7f9ec1a4e700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ebc102aa0 0x7f9ebc0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:37.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.287+0000 7f9ec224f700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ebc104640 0x7f9ebc075370 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4007d10 tx=0x7f9eb4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:37.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.287+0000 7f9eb2ffd700 1 -- 172.21.15.19:0/2815149217 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4021020 con 0x7f9ebc104640 2024-04-25T12:06:37.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.287+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/2815149217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ebc1a2d00 con 0x7f9ebc104640 2024-04-25T12:06:37.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.287+0000 7f9eb2ffd700 1 -- 172.21.15.19:0/2815149217 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9eb401d940 con 0x7f9ebc104640 2024-04-25T12:06:37.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.287+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/2815149217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ebc1a3150 con 0x7f9ebc104640 2024-04-25T12:06:37.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.287+0000 7f9eb2ffd700 1 -- 172.21.15.19:0/2815149217 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4005660 con 0x7f9ebc104640 2024-04-25T12:06:37.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.288+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/2815149217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ebc069730 con 0x7f9ebc104640 2024-04-25T12:06:37.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.289+0000 7f9eb2ffd700 1 -- 172.21.15.19:0/2815149217 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9eb4024020 con 0x7f9ebc104640 2024-04-25T12:06:37.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.290+0000 7f9eb2ffd700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9ea80a3c40 0x7f9ea80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:37.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.290+0000 7f9ec1a4e700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9ea80a3c40 0x7f9ea80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:37.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.290+0000 7f9eb2ffd700 1 -- 172.21.15.19:0/2815149217 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9eb400f020 con 0x7f9ebc104640 2024-04-25T12:06:37.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.291+0000 7f9ec1a4e700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9ea80a3c40 0x7f9ea80a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc103b00 tx=0x7f9eb800a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:37.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.295+0000 7f9eb2ffd700 1 -- 172.21.15.19:0/2815149217 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9eb4009cb0 con 0x7f9ebc104640 2024-04-25T12:06:37.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.522+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/2815149217 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9ebc062e80 con 0x7f9ea80a3c40 2024-04-25T12:06:37.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.523+0000 7f9eb2ffd700 1 -- 172.21.15.19:0/2815149217 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9ebc062e80 con 0x7f9ea80a3c40 2024-04-25T12:06:37.521 INFO:teuthology.orchestra.run.smithi019.stdout:Backend: cephadm 2024-04-25T12:06:37.521 INFO:teuthology.orchestra.run.smithi019.stdout:Available: Yes 2024-04-25T12:06:37.521 INFO:teuthology.orchestra.run.smithi019.stdout:Paused: No 2024-04-25T12:06:37.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.525+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/2815149217 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9ea80a3c40 msgr2=0x7f9ea80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.525+0000 7f9ec3cb2700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9ea80a3c40 0x7f9ea80a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc103b00 tx=0x7f9eb800a680 comp rx=0 tx=0).stop 2024-04-25T12:06:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.525+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/2815149217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ebc104640 msgr2=0x7f9ebc075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.525+0000 7f9ec3cb2700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ebc104640 0x7f9ebc075370 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4007d10 tx=0x7f9eb4007df0 comp rx=0 tx=0).stop 2024-04-25T12:06:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.525+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/2815149217 shutdown_connections 2024-04-25T12:06:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.525+0000 7f9ec3cb2700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9ea80a3c40 0x7f9ea80a6100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.525+0000 7f9ec3cb2700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ebc104640 0x7f9ebc075370 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.525+0000 7f9ec3cb2700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ebc102aa0 0x7f9ebc0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.525+0000 7f9ec3cb2700 1 --2- 172.21.15.19:0/2815149217 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ebc103ca0 0x7f9ebc074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.525+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/2815149217 >> 172.21.15.19:0/2815149217 conn(0x7f9ebc0fe040 msgr2=0x7f9ebc106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.526+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/2815149217 shutdown_connections 2024-04-25T12:06:37.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:37.526+0000 7f9ec3cb2700 1 -- 172.21.15.19:0/2815149217 wait complete. 2024-04-25T12:06:38.282 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T12:06:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:06:39.001 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:38 smithi019 ceph-mon[29213]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T12:06:39.001 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:38 smithi019 ceph-mon[29213]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:39.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:38 smithi169 ceph-mon[32710]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T12:06:39.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:38 smithi169 ceph-mon[32710]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:39.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:38 smithi150 ceph-mon[32250]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-25T12:06:39.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:38 smithi150 ceph-mon[32250]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.337+0000 7f3bdc16e700 1 -- 172.21.15.19:0/970342649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd41011e0 msgr2=0x7f3bd4101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.337+0000 7f3bdc16e700 1 --2- 172.21.15.19:0/970342649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd41011e0 0x7f3bd4101600 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000aad0 tx=0x7f3bd000ade0 comp rx=0 tx=0).stop 2024-04-25T12:06:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.337+0000 7f3bdc16e700 1 -- 172.21.15.19:0/970342649 shutdown_connections 2024-04-25T12:06:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.337+0000 7f3bdc16e700 1 --2- 172.21.15.19:0/970342649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd41011e0 0x7f3bd4101600 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.337+0000 7f3bdc16e700 1 --2- 172.21.15.19:0/970342649 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bd4102d80 0x7f3bd4109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.337+0000 7f3bdc16e700 1 --2- 172.21.15.19:0/970342649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd41023e0 0x7f3bd4102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.337+0000 7f3bdc16e700 1 -- 172.21.15.19:0/970342649 >> 172.21.15.19:0/970342649 conn(0x7f3bd40fc780 msgr2=0x7f3bd40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.337+0000 7f3bdc16e700 1 -- 172.21.15.19:0/970342649 shutdown_connections 2024-04-25T12:06:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.337+0000 7f3bdc16e700 1 -- 172.21.15.19:0/970342649 wait complete. 2024-04-25T12:06:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bdc16e700 1 Processor -- start 2024-04-25T12:06:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bdc16e700 1 -- start start 2024-04-25T12:06:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bdc16e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd41011e0 0x7f3bd4198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bdc16e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bd41023e0 0x7f3bd4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bdc16e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102d80 0x7f3bd41a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bdc16e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd410c540 con 0x7f3bd4102d80 2024-04-25T12:06:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bdc16e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd410c3c0 con 0x7f3bd41023e0 2024-04-25T12:06:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bdc16e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd410c6c0 con 0x7f3bd41011e0 2024-04-25T12:06:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bda70b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102d80 0x7f3bd41a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bda70b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102d80 0x7f3bd41a0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36676/0 (socket says 172.21.15.19:36676) 2024-04-25T12:06:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bda70b700 1 -- 172.21.15.19:0/770415183 learned_addr learned my addr 172.21.15.19:0/770415183 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bd9f0a700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd41011e0 0x7f3bd4198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bd9709700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bd41023e0 0x7f3bd4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bda70b700 1 -- 172.21.15.19:0/770415183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd41011e0 msgr2=0x7f3bd4198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bda70b700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd41011e0 0x7f3bd4198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bda70b700 1 -- 172.21.15.19:0/770415183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bd41023e0 msgr2=0x7f3bd4199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bda70b700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bd41023e0 0x7f3bd4199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.338+0000 7f3bda70b700 1 -- 172.21.15.19:0/770415183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd41a09a0 con 0x7f3bd4102d80 2024-04-25T12:06:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.339+0000 7f3bd9709700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bd41023e0 0x7f3bd4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:06:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.339+0000 7f3bda70b700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102d80 0x7f3bd41a0300 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc007f10 tx=0x7f3bcc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.339+0000 7f3bcaffd700 1 -- 172.21.15.19:0/770415183 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc004bb0 con 0x7f3bd4102d80 2024-04-25T12:06:40.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.339+0000 7f3bcaffd700 1 -- 172.21.15.19:0/770415183 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3bcc004d50 con 0x7f3bd4102d80 2024-04-25T12:06:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.339+0000 7f3bdc16e700 1 -- 172.21.15.19:0/770415183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd41a0c90 con 0x7f3bd4102d80 2024-04-25T12:06:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.339+0000 7f3bcaffd700 1 -- 172.21.15.19:0/770415183 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc0133f0 con 0x7f3bd4102d80 2024-04-25T12:06:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.339+0000 7f3bdc16e700 1 -- 172.21.15.19:0/770415183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd41a1180 con 0x7f3bd4102d80 2024-04-25T12:06:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.340+0000 7f3bdc16e700 1 -- 172.21.15.19:0/770415183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd4069730 con 0x7f3bd4102d80 2024-04-25T12:06:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.341+0000 7f3bcaffd700 1 -- 172.21.15.19:0/770415183 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3bcc01a020 con 0x7f3bd4102d80 2024-04-25T12:06:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.342+0000 7f3bcaffd700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3bc00a3c40 0x7f3bc00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.342+0000 7f3bd9f0a700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3bc00a3c40 0x7f3bc00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.342+0000 7f3bcaffd700 1 -- 172.21.15.19:0/770415183 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f3bcc0d02a0 con 0x7f3bd4102d80 2024-04-25T12:06:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.342+0000 7f3bd9f0a700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3bc00a3c40 0x7f3bc00a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000aaa0 tx=0x7f3bd0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:40.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.346+0000 7f3bcaffd700 1 -- 172.21.15.19:0/770415183 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3bcc010070 con 0x7f3bd4102d80 2024-04-25T12:06:40.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.565+0000 7f3bdc16e700 1 -- 172.21.15.19:0/770415183 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f3bd4109010 con 0x7f3bc00a3c40 2024-04-25T12:06:40.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.579+0000 7f3bcaffd700 1 -- 172.21.15.19:0/770415183 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f3bd4109010 con 0x7f3bc00a3c40 2024-04-25T12:06:40.577 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T12:06:40.577 INFO:teuthology.orchestra.run.smithi019.stdout:mds.a smithi019 running (106s) 18s ago 105s 17.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 765dadd9a116 2024-04-25T12:06:40.578 INFO:teuthology.orchestra.run.smithi019.stdout:mds.b smithi150 running (102s) 35s ago 102s 19.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 764b69f2b938 2024-04-25T12:06:40.578 INFO:teuthology.orchestra.run.smithi019.stdout:mds.c smithi169 running (117s) 36s ago 117s 20.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 c14b15ecb108 2024-04-25T12:06:40.578 INFO:teuthology.orchestra.run.smithi019.stdout:mds.d smithi019 running (2m) 18s ago 2m 15.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3d49cfc6549a 2024-04-25T12:06:40.578 INFO:teuthology.orchestra.run.smithi019.stdout:mds.e smithi150 running (2m) 35s ago 2m 16.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 58a4be0bae32 2024-04-25T12:06:40.578 INFO:teuthology.orchestra.run.smithi019.stdout:mds.f smithi169 running (2m) 36s ago 119s 17.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3ac46dda412e 2024-04-25T12:06:40.578 INFO:teuthology.orchestra.run.smithi019.stdout:mds.g smithi019 running (108s) 18s ago 108s 16.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ad1bcb299205 2024-04-25T12:06:40.578 INFO:teuthology.orchestra.run.smithi019.stdout:mds.h smithi150 running (114s) 35s ago 114s 17.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3c0afcfe116f 2024-04-25T12:06:40.578 INFO:teuthology.orchestra.run.smithi019.stdout:mds.i smithi169 running (111s) 36s ago 111s 26.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 bdf8b83936cf 2024-04-25T12:06:40.578 INFO:teuthology.orchestra.run.smithi019.stdout:mds.j smithi019 running (2m) 18s ago 2m 12.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 66947b32c094 2024-04-25T12:06:40.578 INFO:teuthology.orchestra.run.smithi019.stdout:mds.k smithi150 running (2m) 35s ago 2m 19.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 0cf449dd81c5 2024-04-25T12:06:40.578 INFO:teuthology.orchestra.run.smithi019.stdout:mds.l smithi169 running (2m) 36s ago 2m 17.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 49e057e4f520 2024-04-25T12:06:40.578 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.x smithi019 *:9283,8765 running (9m) 18s ago 9m 514M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6340be20f2fa 2024-04-25T12:06:40.578 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.y smithi150 *:8443,8765 running (7m) 35s ago 7m 453M - 19.0.0-3156-gd69f13d8 da4b004c09e8 77cdcfc15951 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.z smithi169 *:8443,8765 running (7m) 36s ago 7m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 631bf7fcd2a3 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:mon.a smithi019 running (9m) 18s ago 9m 50.9M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 4a9fc10a4dca 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:mon.b smithi150 running (7m) 35s ago 7m 38.6M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 2062640205a6 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:mon.c smithi169 running (8m) 36s ago 7m 40.0M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 e2e3dded6e41 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (7m) 18s ago 7m 45.4M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 647cfc1df8a8 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi150 running (6m) 35s ago 6m 57.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 580f7c13926e 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi169 running (6m) 36s ago 6m 48.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 2e96b3b5f201 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi019 running (6m) 18s ago 6m 55.1M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 afa45ec27c7d 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi150 running (5m) 35s ago 5m 44.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 7dc2434adb5a 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi169 running (5m) 36s ago 5m 57.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 eea1c1d1e37d 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 smithi019 running (4m) 18s ago 4m 42.0M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 24e9f2ad931d 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 smithi150 running (4m) 35s ago 4m 54.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 831b179dfb22 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 smithi169 running (3m) 36s ago 3m 54.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 ed3ec86631bc 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 smithi019 running (3m) 18s ago 3m 54.2M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 b6ee6182fab8 2024-04-25T12:06:40.579 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 smithi150 running (3m) 35s ago 2m 54.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 17d5614afcf5 2024-04-25T12:06:40.580 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 smithi169 running (2m) 36s ago 2m 41.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 8d4c3df65cae 2024-04-25T12:06:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.581+0000 7f3bdc16e700 1 -- 172.21.15.19:0/770415183 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3bc00a3c40 msgr2=0x7f3bc00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.581+0000 7f3bdc16e700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3bc00a3c40 0x7f3bc00a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000aaa0 tx=0x7f3bd0005fb0 comp rx=0 tx=0).stop 2024-04-25T12:06:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.581+0000 7f3bdc16e700 1 -- 172.21.15.19:0/770415183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102d80 msgr2=0x7f3bd41a0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.581+0000 7f3bdc16e700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102d80 0x7f3bd41a0300 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc007f10 tx=0x7f3bcc00d3b0 comp rx=0 tx=0).stop 2024-04-25T12:06:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.581+0000 7f3bdc16e700 1 -- 172.21.15.19:0/770415183 shutdown_connections 2024-04-25T12:06:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.581+0000 7f3bdc16e700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3bc00a3c40 0x7f3bc00a6100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.581+0000 7f3bdc16e700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bd4102d80 0x7f3bd41a0300 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.581+0000 7f3bdc16e700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3bd41011e0 0x7f3bd4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.581+0000 7f3bdc16e700 1 --2- 172.21.15.19:0/770415183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bd41023e0 0x7f3bd4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.581+0000 7f3bdc16e700 1 -- 172.21.15.19:0/770415183 >> 172.21.15.19:0/770415183 conn(0x7f3bd40fc780 msgr2=0x7f3bd41078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.582+0000 7f3bdc16e700 1 -- 172.21.15.19:0/770415183 shutdown_connections 2024-04-25T12:06:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:40.582+0000 7f3bdc16e700 1 -- 172.21.15.19:0/770415183 wait complete. 2024-04-25T12:06:40.966 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:40 smithi150 ceph-mon[32250]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T12:06:40.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:40 smithi019 ceph-mon[29213]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T12:06:41.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:40 smithi169 ceph-mon[32710]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T12:06:41.387 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T12:06:41.638 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:06:42.002 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:41 smithi019 ceph-mon[29213]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:42.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:41 smithi169 ceph-mon[32710]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:42.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:41 smithi150 ceph-mon[32250]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:42.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:42 smithi019 ceph-mon[29213]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T12:06:43.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:42 smithi169 ceph-mon[32710]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T12:06:43.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:42 smithi150 ceph-mon[32250]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T12:06:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.288+0000 7fd28e1ca700 1 -- 172.21.15.19:0/3132084064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd288105e80 msgr2=0x7fd288108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.288+0000 7fd28e1ca700 1 --2- 172.21.15.19:0/3132084064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd288105e80 0x7fd288108270 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009990 tx=0x7fd27c009ca0 comp rx=0 tx=0).stop 2024-04-25T12:06:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.288+0000 7fd28e1ca700 1 -- 172.21.15.19:0/3132084064 shutdown_connections 2024-04-25T12:06:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.288+0000 7fd28e1ca700 1 --2- 172.21.15.19:0/3132084064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2881087b0 0x7fd28810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.288+0000 7fd28e1ca700 1 --2- 172.21.15.19:0/3132084064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd288103550 0x7fd288105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.288+0000 7fd28e1ca700 1 --2- 172.21.15.19:0/3132084064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd288105e80 0x7fd288108270 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:43.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.288+0000 7fd28e1ca700 1 -- 172.21.15.19:0/3132084064 >> 172.21.15.19:0/3132084064 conn(0x7fd2880fcf60 msgr2=0x7fd2880ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.288+0000 7fd28e1ca700 1 -- 172.21.15.19:0/3132084064 shutdown_connections 2024-04-25T12:06:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.288+0000 7fd28e1ca700 1 -- 172.21.15.19:0/3132084064 wait complete. 2024-04-25T12:06:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.288+0000 7fd28e1ca700 1 Processor -- start 2024-04-25T12:06:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.288+0000 7fd28e1ca700 1 -- start start 2024-04-25T12:06:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd28e1ca700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd288103550 0x7fd288198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd28e1ca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd288105e80 0x7fd288199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd28e1ca700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2881087b0 0x7fd2881a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd28e1ca700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd28810e780 con 0x7fd2881087b0 2024-04-25T12:06:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd28e1ca700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd28810e600 con 0x7fd288105e80 2024-04-25T12:06:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd28e1ca700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd28810e900 con 0x7fd288103550 2024-04-25T12:06:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd287fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2881087b0 0x7fd2881a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd287fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2881087b0 0x7fd2881a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35938/0 (socket says 172.21.15.19:35938) 2024-04-25T12:06:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd287fff700 1 -- 172.21.15.19:0/1392211007 learned_addr learned my addr 172.21.15.19:0/1392211007 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd2877fe700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd288103550 0x7fd288198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd286ffd700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd288105e80 0x7fd288199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd287fff700 1 -- 172.21.15.19:0/1392211007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd288103550 msgr2=0x7fd288198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd287fff700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd288103550 0x7fd288198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd287fff700 1 -- 172.21.15.19:0/1392211007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd288105e80 msgr2=0x7fd288199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd287fff700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd288105e80 0x7fd288199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd287fff700 1 -- 172.21.15.19:0/1392211007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2881a0a40 con 0x7fd2881087b0 2024-04-25T12:06:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd286ffd700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd288105e80 0x7fd288199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:06:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd287fff700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2881087b0 0x7fd2881a03a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd28000ba10 tx=0x7fd28000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd284ff9700 1 -- 172.21.15.19:0/1392211007 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd280019070 con 0x7fd2881087b0 2024-04-25T12:06:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd28e1ca700 1 -- 172.21.15.19:0/1392211007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2881a0d30 con 0x7fd2881087b0 2024-04-25T12:06:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd284ff9700 1 -- 172.21.15.19:0/1392211007 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd280014440 con 0x7fd2881087b0 2024-04-25T12:06:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.289+0000 7fd284ff9700 1 -- 172.21.15.19:0/1392211007 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2800125a0 con 0x7fd2881087b0 2024-04-25T12:06:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.290+0000 7fd28e1ca700 1 -- 172.21.15.19:0/1392211007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2881a1240 con 0x7fd2881087b0 2024-04-25T12:06:43.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.291+0000 7fd28e1ca700 1 -- 172.21.15.19:0/1392211007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd288069730 con 0x7fd2881087b0 2024-04-25T12:06:43.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.292+0000 7fd284ff9700 1 -- 172.21.15.19:0/1392211007 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd28001c070 con 0x7fd2881087b0 2024-04-25T12:06:43.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.293+0000 7fd284ff9700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd2700a3c40 0x7fd2700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:43.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.293+0000 7fd2877fe700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd2700a3c40 0x7fd2700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:43.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.293+0000 7fd284ff9700 1 -- 172.21.15.19:0/1392211007 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd2800cfca0 con 0x7fd2881087b0 2024-04-25T12:06:43.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.293+0000 7fd2877fe700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd2700a3c40 0x7fd2700a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd278006fd0 tx=0x7fd278008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:43.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.297+0000 7fd284ff9700 1 -- 172.21.15.19:0/1392211007 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd28009c6f0 con 0x7fd2881087b0 2024-04-25T12:06:43.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.515+0000 7fd28e1ca700 1 -- 172.21.15.19:0/1392211007 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fd288062e80 con 0x7fd2700a3c40 2024-04-25T12:06:43.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.523+0000 7fd284ff9700 1 -- 172.21.15.19:0/1392211007 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fd288062e80 con 0x7fd2700a3c40 2024-04-25T12:06:43.521 INFO:teuthology.orchestra.run.smithi019.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T12:06:43.521 INFO:teuthology.orchestra.run.smithi019.stdout:mds.all 12/12 39s ago 2m smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi169=c;smithi169=f;smithi169=i;smithi169=l;count:12 2024-04-25T12:06:43.521 INFO:teuthology.orchestra.run.smithi019.stdout:mgr 3/3 39s ago 7m smithi019=x;smithi150=y;smithi169=z;count:3 2024-04-25T12:06:43.522 INFO:teuthology.orchestra.run.smithi019.stdout:mon 3/3 39s ago 8m smithi019:172.21.15.19=a;smithi150:172.21.15.150=b;smithi169:172.21.15.169=c;count:3 2024-04-25T12:06:43.522 INFO:teuthology.orchestra.run.smithi019.stdout:osd 12 39s ago - 2024-04-25T12:06:43.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.525+0000 7fd28e1ca700 1 -- 172.21.15.19:0/1392211007 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd2700a3c40 msgr2=0x7fd2700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:43.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.525+0000 7fd28e1ca700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd2700a3c40 0x7fd2700a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd278006fd0 tx=0x7fd278008040 comp rx=0 tx=0).stop 2024-04-25T12:06:43.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.525+0000 7fd28e1ca700 1 -- 172.21.15.19:0/1392211007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2881087b0 msgr2=0x7fd2881a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:43.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.525+0000 7fd28e1ca700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2881087b0 0x7fd2881a03a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd28000ba10 tx=0x7fd28000bdd0 comp rx=0 tx=0).stop 2024-04-25T12:06:43.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.525+0000 7fd28e1ca700 1 -- 172.21.15.19:0/1392211007 shutdown_connections 2024-04-25T12:06:43.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.525+0000 7fd28e1ca700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd2700a3c40 0x7fd2700a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:43.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.525+0000 7fd28e1ca700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2881087b0 0x7fd2881a03a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:43.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.525+0000 7fd28e1ca700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd288103550 0x7fd288198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:43.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.525+0000 7fd28e1ca700 1 --2- 172.21.15.19:0/1392211007 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd288105e80 0x7fd288199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:43.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.525+0000 7fd28e1ca700 1 -- 172.21.15.19:0/1392211007 >> 172.21.15.19:0/1392211007 conn(0x7fd2880fcf60 msgr2=0x7fd288103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:43.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.525+0000 7fd28e1ca700 1 -- 172.21.15.19:0/1392211007 shutdown_connections 2024-04-25T12:06:43.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:43.525+0000 7fd28e1ca700 1 -- 172.21.15.19:0/1392211007 wait complete. 2024-04-25T12:06:44.232 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T12:06:44.485 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:06:44.877 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:44 smithi019 ceph-mon[29213]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T12:06:44.878 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:44 smithi019 ceph-mon[29213]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:45.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:44 smithi169 ceph-mon[32710]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T12:06:45.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:44 smithi169 ceph-mon[32710]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:45.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:44 smithi150 ceph-mon[32250]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T12:06:45.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:44 smithi150 ceph-mon[32250]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:46.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.222+0000 7f88e0f98700 1 -- 172.21.15.19:0/3450003718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88dc103530 msgr2=0x7f88dc105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.222+0000 7f88e0f98700 1 --2- 172.21.15.19:0/3450003718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88dc103530 0x7f88dc105920 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009990 tx=0x7f88cc009ca0 comp rx=0 tx=0).stop 2024-04-25T12:06:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.222+0000 7f88e0f98700 1 -- 172.21.15.19:0/3450003718 shutdown_connections 2024-04-25T12:06:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.222+0000 7f88e0f98700 1 --2- 172.21.15.19:0/3450003718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88dc103530 0x7f88dc105920 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.222+0000 7f88e0f98700 1 --2- 172.21.15.19:0/3450003718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88dc108790 0x7f88dc10ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.222+0000 7f88e0f98700 1 --2- 172.21.15.19:0/3450003718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88dc105e60 0x7f88dc108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.222+0000 7f88e0f98700 1 -- 172.21.15.19:0/3450003718 >> 172.21.15.19:0/3450003718 conn(0x7f88dc0fcf40 msgr2=0x7f88dc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.222+0000 7f88e0f98700 1 -- 172.21.15.19:0/3450003718 shutdown_connections 2024-04-25T12:06:46.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.222+0000 7f88e0f98700 1 -- 172.21.15.19:0/3450003718 wait complete. 2024-04-25T12:06:46.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.223+0000 7f88e0f98700 1 Processor -- start 2024-04-25T12:06:46.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.223+0000 7f88e0f98700 1 -- start start 2024-04-25T12:06:46.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.223+0000 7f88e0f98700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88dc103530 0x7f88dc198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:46.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.223+0000 7f88e0f98700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88dc105e60 0x7f88dc199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:46.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.223+0000 7f88e0f98700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88dc108790 0x7f88dc1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.223+0000 7f88e0f98700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc10e660 con 0x7f88dc105e60 2024-04-25T12:06:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.223+0000 7f88e0f98700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc10e4e0 con 0x7f88dc103530 2024-04-25T12:06:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.223+0000 7f88e0f98700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc10e7e0 con 0x7f88dc108790 2024-04-25T12:06:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.223+0000 7f88d9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88dc105e60 0x7f88dc199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.223+0000 7f88d9d9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88dc105e60 0x7f88dc199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35950/0 (socket says 172.21.15.19:35950) 2024-04-25T12:06:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.223+0000 7f88d9d9b700 1 -- 172.21.15.19:0/1340492163 learned_addr learned my addr 172.21.15.19:0/1340492163 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.223+0000 7f88dad9d700 1 --2- 172.21.15.19:0/1340492163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88dc108790 0x7f88dc1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.223+0000 7f88da59c700 1 --2- 172.21.15.19:0/1340492163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88dc103530 0x7f88dc198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.224+0000 7f88d9d9b700 1 -- 172.21.15.19:0/1340492163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88dc108790 msgr2=0x7f88dc1a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.224+0000 7f88d9d9b700 1 --2- 172.21.15.19:0/1340492163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88dc108790 0x7f88dc1a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.224+0000 7f88d9d9b700 1 -- 172.21.15.19:0/1340492163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88dc103530 msgr2=0x7f88dc198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:46.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.224+0000 7f88d9d9b700 1 --2- 172.21.15.19:0/1340492163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88dc103530 0x7f88dc198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:46.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.224+0000 7f88d9d9b700 1 -- 172.21.15.19:0/1340492163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc1a0990 con 0x7f88dc105e60 2024-04-25T12:06:46.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.224+0000 7f88d9d9b700 1 --2- 172.21.15.19:0/1340492163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88dc105e60 0x7f88dc199270 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f88d000d980 tx=0x7f88d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:46.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.224+0000 7f88cb7fe700 1 -- 172.21.15.19:0/1340492163 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0016070 con 0x7f88dc105e60 2024-04-25T12:06:46.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.224+0000 7f88e0f98700 1 -- 172.21.15.19:0/1340492163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88dc1a0c80 con 0x7f88dc105e60 2024-04-25T12:06:46.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.224+0000 7f88cb7fe700 1 -- 172.21.15.19:0/1340492163 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f88d0009da0 con 0x7f88dc105e60 2024-04-25T12:06:46.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.224+0000 7f88cb7fe700 1 -- 172.21.15.19:0/1340492163 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d000f3c0 con 0x7f88dc105e60 2024-04-25T12:06:46.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.224+0000 7f88e0f98700 1 -- 172.21.15.19:0/1340492163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88dc1a11c0 con 0x7f88dc105e60 2024-04-25T12:06:46.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.225+0000 7f88e0f98700 1 -- 172.21.15.19:0/1340492163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88dc069730 con 0x7f88dc105e60 2024-04-25T12:06:46.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.226+0000 7f88cb7fe700 1 -- 172.21.15.19:0/1340492163 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f88d0019070 con 0x7f88dc105e60 2024-04-25T12:06:46.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.227+0000 7f88cb7fe700 1 --2- 172.21.15.19:0/1340492163 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88c40a3c40 0x7f88c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:46.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.227+0000 7f88da59c700 1 --2- 172.21.15.19:0/1340492163 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88c40a3c40 0x7f88c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:46.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.227+0000 7f88cb7fe700 1 -- 172.21.15.19:0/1340492163 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f88d00d02b0 con 0x7f88dc105e60 2024-04-25T12:06:46.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.227+0000 7f88da59c700 1 --2- 172.21.15.19:0/1340492163 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88c40a3c40 0x7f88c40a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00b5c0 tx=0x7f88cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:46.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.232+0000 7f88cb7fe700 1 -- 172.21.15.19:0/1340492163 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88d009cc10 con 0x7f88dc105e60 2024-04-25T12:06:46.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.444+0000 7f88e0f98700 1 -- 172.21.15.19:0/1340492163 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f88dc062e80 con 0x7f88c40a3c40 2024-04-25T12:06:46.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.446+0000 7f88cb7fe700 1 -- 172.21.15.19:0/1340492163 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f88dc062e80 con 0x7f88c40a3c40 2024-04-25T12:06:46.444 INFO:teuthology.orchestra.run.smithi019.stdout:HOST ADDR LABELS STATUS 2024-04-25T12:06:46.444 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 172.21.15.19 2024-04-25T12:06:46.444 INFO:teuthology.orchestra.run.smithi019.stdout:smithi150 172.21.15.150 2024-04-25T12:06:46.444 INFO:teuthology.orchestra.run.smithi019.stdout:smithi169 172.21.15.169 2024-04-25T12:06:46.444 INFO:teuthology.orchestra.run.smithi019.stdout:3 hosts in cluster 2024-04-25T12:06:46.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.448+0000 7f88e0f98700 1 -- 172.21.15.19:0/1340492163 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88c40a3c40 msgr2=0x7f88c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:46.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.448+0000 7f88e0f98700 1 --2- 172.21.15.19:0/1340492163 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88c40a3c40 0x7f88c40a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00b5c0 tx=0x7f88cc005fb0 comp rx=0 tx=0).stop 2024-04-25T12:06:46.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.448+0000 7f88e0f98700 1 -- 172.21.15.19:0/1340492163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88dc105e60 msgr2=0x7f88dc199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:46.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.448+0000 7f88e0f98700 1 --2- 172.21.15.19:0/1340492163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88dc105e60 0x7f88dc199270 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f88d000d980 tx=0x7f88d000dc90 comp rx=0 tx=0).stop 2024-04-25T12:06:46.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.448+0000 7f88e0f98700 1 -- 172.21.15.19:0/1340492163 shutdown_connections 2024-04-25T12:06:46.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.448+0000 7f88e0f98700 1 --2- 172.21.15.19:0/1340492163 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f88c40a3c40 0x7f88c40a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:46.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.448+0000 7f88e0f98700 1 --2- 172.21.15.19:0/1340492163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88dc108790 0x7f88dc1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:46.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.448+0000 7f88e0f98700 1 --2- 172.21.15.19:0/1340492163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88dc103530 0x7f88dc198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:46.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.448+0000 7f88e0f98700 1 --2- 172.21.15.19:0/1340492163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88dc105e60 0x7f88dc199270 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:46.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.448+0000 7f88e0f98700 1 -- 172.21.15.19:0/1340492163 >> 172.21.15.19:0/1340492163 conn(0x7f88dc0fcf40 msgr2=0x7f88dc103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:46.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.448+0000 7f88e0f98700 1 -- 172.21.15.19:0/1340492163 shutdown_connections 2024-04-25T12:06:46.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:46.448+0000 7f88e0f98700 1 -- 172.21.15.19:0/1340492163 wait complete. 2024-04-25T12:06:47.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:46 smithi019 ceph-mon[29213]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:06:47.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:46 smithi169 ceph-mon[32710]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:06:47.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:46 smithi150 ceph-mon[32250]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:06:47.204 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T12:06:47.461 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:06:47.854 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:47 smithi019 ceph-mon[29213]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:48.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:47 smithi169 ceph-mon[32710]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:48.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:47 smithi150 ceph-mon[32250]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:48.891 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:48 smithi019 ceph-mon[29213]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T12:06:49.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:48 smithi169 ceph-mon[32710]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T12:06:49.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:48 smithi150 ceph-mon[32250]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T12:06:49.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.321+0000 7f07e4cd8700 1 -- 172.21.15.19:0/3857039055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e0103cc0 msgr2=0x7f07e0104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:49.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.321+0000 7f07e4cd8700 1 --2- 172.21.15.19:0/3857039055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e0103cc0 0x7f07e0104120 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f07d40099c0 tx=0x7f07d4009cd0 comp rx=0 tx=0).stop 2024-04-25T12:06:49.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.321+0000 7f07e4cd8700 1 -- 172.21.15.19:0/3857039055 shutdown_connections 2024-04-25T12:06:49.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.321+0000 7f07e4cd8700 1 --2- 172.21.15.19:0/3857039055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e0104660 0x7f07e010af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:49.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.321+0000 7f07e4cd8700 1 --2- 172.21.15.19:0/3857039055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07e0102ac0 0x7f07e0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:49.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.321+0000 7f07e4cd8700 1 --2- 172.21.15.19:0/3857039055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e0103cc0 0x7f07e0104120 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:49.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.321+0000 7f07e4cd8700 1 -- 172.21.15.19:0/3857039055 >> 172.21.15.19:0/3857039055 conn(0x7f07e00fe060 msgr2=0x7f07e01004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:49.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.321+0000 7f07e4cd8700 1 -- 172.21.15.19:0/3857039055 shutdown_connections 2024-04-25T12:06:49.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.321+0000 7f07e4cd8700 1 -- 172.21.15.19:0/3857039055 wait complete. 2024-04-25T12:06:49.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07e4cd8700 1 Processor -- start 2024-04-25T12:06:49.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07e4cd8700 1 -- start start 2024-04-25T12:06:49.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07e4cd8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07e0102ac0 0x7f07e00767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:49.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07e4cd8700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e0103cc0 0x7f07e0074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:49.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07e4cd8700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e0104660 0x7f07e0075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:49.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07e4cd8700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07e010e6a0 con 0x7f07e0103cc0 2024-04-25T12:06:49.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07e4cd8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f07e010e520 con 0x7f07e0102ac0 2024-04-25T12:06:49.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07e4cd8700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f07e010e820 con 0x7f07e0104660 2024-04-25T12:06:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07ddd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e0103cc0 0x7f07e0074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07ddd9b700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e0103cc0 0x7f07e0074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35964/0 (socket says 172.21.15.19:35964) 2024-04-25T12:06:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07ddd9b700 1 -- 172.21.15.19:0/2000778864 learned_addr learned my addr 172.21.15.19:0/2000778864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07ded9d700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e0104660 0x7f07e0075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07de59c700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07e0102ac0 0x7f07e00767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07ddd9b700 1 -- 172.21.15.19:0/2000778864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e0104660 msgr2=0x7f07e0075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07ddd9b700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e0104660 0x7f07e0075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07ddd9b700 1 -- 172.21.15.19:0/2000778864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07e0102ac0 msgr2=0x7f07e00767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07ddd9b700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07e0102ac0 0x7f07e00767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07ddd9b700 1 -- 172.21.15.19:0/2000778864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e0075c20 con 0x7f07e0103cc0 2024-04-25T12:06:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07de59c700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07e0102ac0 0x7f07e00767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.322+0000 7f07ddd9b700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e0103cc0 0x7f07e0074e30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009f50 tx=0x7f07d4004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:49.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.323+0000 7f07cf7fe700 1 -- 172.21.15.19:0/2000778864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4003c40 con 0x7f07e0103cc0 2024-04-25T12:06:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.323+0000 7f07cf7fe700 1 -- 172.21.15.19:0/2000778864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f07d4004800 con 0x7f07e0103cc0 2024-04-25T12:06:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.323+0000 7f07e4cd8700 1 -- 172.21.15.19:0/2000778864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e01a2d80 con 0x7f07e0103cc0 2024-04-25T12:06:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.323+0000 7f07cf7fe700 1 -- 172.21.15.19:0/2000778864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4005380 con 0x7f07e0103cc0 2024-04-25T12:06:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.323+0000 7f07e4cd8700 1 -- 172.21.15.19:0/2000778864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e01a31b0 con 0x7f07e0103cc0 2024-04-25T12:06:49.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.324+0000 7f07e4cd8700 1 -- 172.21.15.19:0/2000778864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e002df80 con 0x7f07e0103cc0 2024-04-25T12:06:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.325+0000 7f07cf7fe700 1 -- 172.21.15.19:0/2000778864 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f07d4028060 con 0x7f07e0103cc0 2024-04-25T12:06:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.326+0000 7f07cf7fe700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f07c80a3cc0 0x7f07c80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.326+0000 7f07de59c700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f07c80a3cc0 0x7f07c80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.326+0000 7f07cf7fe700 1 -- 172.21.15.19:0/2000778864 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f07d40d0da0 con 0x7f07e0103cc0 2024-04-25T12:06:49.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.326+0000 7f07de59c700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f07c80a3cc0 0x7f07c80a6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f07d0005950 tx=0x7f07d00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:49.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.330+0000 7f07cf7fe700 1 -- 172.21.15.19:0/2000778864 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07d405cff0 con 0x7f07e0103cc0 2024-04-25T12:06:49.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.548+0000 7f07e4cd8700 1 -- 172.21.15.19:0/2000778864 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f07e0062e80 con 0x7f07c80a3cc0 2024-04-25T12:06:49.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.549+0000 7f07cf7fe700 1 -- 172.21.15.19:0/2000778864 <== mgr.14152 v2:172.21.15.19:6800/2853466745 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f07e0062e80 con 0x7f07c80a3cc0 2024-04-25T12:06:49.548 INFO:teuthology.orchestra.run.smithi019.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T12:06:49.548 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330002G400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T12:06:49.548 INFO:teuthology.orchestra.run.smithi019.stdout:smithi150 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GX400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T12:06:49.548 INFO:teuthology.orchestra.run.smithi019.stdout:smithi169 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300LN400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T12:06:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.552+0000 7f07e4cd8700 1 -- 172.21.15.19:0/2000778864 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f07c80a3cc0 msgr2=0x7f07c80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.552+0000 7f07e4cd8700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f07c80a3cc0 0x7f07c80a6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f07d0005950 tx=0x7f07d00058e0 comp rx=0 tx=0).stop 2024-04-25T12:06:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.552+0000 7f07e4cd8700 1 -- 172.21.15.19:0/2000778864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e0103cc0 msgr2=0x7f07e0074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.552+0000 7f07e4cd8700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e0103cc0 0x7f07e0074e30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009f50 tx=0x7f07d4004790 comp rx=0 tx=0).stop 2024-04-25T12:06:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.552+0000 7f07e4cd8700 1 -- 172.21.15.19:0/2000778864 shutdown_connections 2024-04-25T12:06:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.552+0000 7f07e4cd8700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f07c80a3cc0 0x7f07c80a6180 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.552+0000 7f07e4cd8700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f07e0104660 0x7f07e0075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.552+0000 7f07e4cd8700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07e0102ac0 0x7f07e00767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.552+0000 7f07e4cd8700 1 --2- 172.21.15.19:0/2000778864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07e0103cc0 0x7f07e0074e30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.552+0000 7f07e4cd8700 1 -- 172.21.15.19:0/2000778864 >> 172.21.15.19:0/2000778864 conn(0x7f07e00fe060 msgr2=0x7f07e0106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.553+0000 7f07e4cd8700 1 -- 172.21.15.19:0/2000778864 shutdown_connections 2024-04-25T12:06:49.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:49.553+0000 7f07e4cd8700 1 -- 172.21.15.19:0/2000778864 wait complete. 2024-04-25T12:06:50.370 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T12:06:50.379 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T12:06:50.379 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T12:06:50.636 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:06:50.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:50 smithi019 ceph-mon[29213]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:06:50.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:50 smithi019 ceph-mon[29213]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:51.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:50 smithi169 ceph-mon[32710]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:06:51.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:50 smithi169 ceph-mon[32710]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:51.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:50 smithi150 ceph-mon[32250]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:06:51.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:50 smithi150 ceph-mon[32250]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:06:52.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.522+0000 7f877582e700 1 -- 172.21.15.19:0/3611096376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8770102ab0 msgr2=0x7f8770102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:52.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.522+0000 7f877582e700 1 --2- 172.21.15.19:0/3611096376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8770102ab0 0x7f8770102ed0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f87600099c0 tx=0x7f8760009cd0 comp rx=0 tx=0).stop 2024-04-25T12:06:52.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.522+0000 7f877582e700 1 -- 172.21.15.19:0/3611096376 shutdown_connections 2024-04-25T12:06:52.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.522+0000 7f877582e700 1 --2- 172.21.15.19:0/3611096376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8770102ab0 0x7f8770102ed0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:52.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.522+0000 7f877582e700 1 --2- 172.21.15.19:0/3611096376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8770104650 0x7f877010af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:52.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.522+0000 7f877582e700 1 --2- 172.21.15.19:0/3611096376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8770103cb0 0x7f8770104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:52.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.522+0000 7f877582e700 1 -- 172.21.15.19:0/3611096376 >> 172.21.15.19:0/3611096376 conn(0x7f87700fe050 msgr2=0x7f8770100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:52.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.522+0000 7f877582e700 1 -- 172.21.15.19:0/3611096376 shutdown_connections 2024-04-25T12:06:52.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.522+0000 7f877582e700 1 -- 172.21.15.19:0/3611096376 wait complete. 2024-04-25T12:06:52.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.523+0000 7f877582e700 1 Processor -- start 2024-04-25T12:06:52.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.523+0000 7f877582e700 1 -- start start 2024-04-25T12:06:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.523+0000 7f877582e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8770102ab0 0x7f8770198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.523+0000 7f877582e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8770103cb0 0x7f8770199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.523+0000 7f877582e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8770104650 0x7f87701a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.523+0000 7f877582e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f877010e700 con 0x7f8770103cb0 2024-04-25T12:06:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.523+0000 7f877582e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f877010e580 con 0x7f8770104650 2024-04-25T12:06:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.523+0000 7f877582e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f877010e880 con 0x7f8770102ab0 2024-04-25T12:06:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.523+0000 7f876e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8770103cb0 0x7f8770199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.523+0000 7f876e7fc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8770103cb0 0x7f8770199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55140/0 (socket says 172.21.15.19:55140) 2024-04-25T12:06:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.523+0000 7f876e7fc700 1 -- 172.21.15.19:0/2165689998 learned_addr learned my addr 172.21.15.19:0/2165689998 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.523+0000 7f876effd700 1 --2- 172.21.15.19:0/2165689998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8770102ab0 0x7f8770198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.523+0000 7f876f7fe700 1 --2- 172.21.15.19:0/2165689998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8770104650 0x7f87701a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.524+0000 7f876e7fc700 1 -- 172.21.15.19:0/2165689998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8770102ab0 msgr2=0x7f8770198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.524+0000 7f876e7fc700 1 --2- 172.21.15.19:0/2165689998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8770102ab0 0x7f8770198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:52.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.524+0000 7f876e7fc700 1 -- 172.21.15.19:0/2165689998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8770104650 msgr2=0x7f87701a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:52.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.524+0000 7f876e7fc700 1 --2- 172.21.15.19:0/2165689998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8770104650 0x7f87701a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:52.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.524+0000 7f876e7fc700 1 -- 172.21.15.19:0/2165689998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87701a0950 con 0x7f8770103cb0 2024-04-25T12:06:52.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.524+0000 7f876e7fc700 1 --2- 172.21.15.19:0/2165689998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8770103cb0 0x7f8770199240 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f875800d980 tx=0x7f875800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:52.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.524+0000 7f877482c700 1 -- 172.21.15.19:0/2165689998 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758017070 con 0x7f8770103cb0 2024-04-25T12:06:52.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.524+0000 7f877582e700 1 -- 172.21.15.19:0/2165689998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87701a0c40 con 0x7f8770103cb0 2024-04-25T12:06:52.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.524+0000 7f877482c700 1 -- 172.21.15.19:0/2165689998 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8758005020 con 0x7f8770103cb0 2024-04-25T12:06:52.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.524+0000 7f877482c700 1 -- 172.21.15.19:0/2165689998 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875800fb30 con 0x7f8770103cb0 2024-04-25T12:06:52.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.524+0000 7f877582e700 1 -- 172.21.15.19:0/2165689998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87701a1150 con 0x7f8770103cb0 2024-04-25T12:06:52.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.525+0000 7f877582e700 1 -- 172.21.15.19:0/2165689998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8770069730 con 0x7f8770103cb0 2024-04-25T12:06:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.526+0000 7f877482c700 1 -- 172.21.15.19:0/2165689998 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f87580051d0 con 0x7f8770103cb0 2024-04-25T12:06:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.527+0000 7f877482c700 1 --2- 172.21.15.19:0/2165689998 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f875c0a3c40 0x7f875c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.527+0000 7f876effd700 1 --2- 172.21.15.19:0/2165689998 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f875c0a3c40 0x7f875c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.527+0000 7f877482c700 1 -- 172.21.15.19:0/2165689998 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f87580d0290 con 0x7f8770103cb0 2024-04-25T12:06:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.527+0000 7f876effd700 1 --2- 172.21.15.19:0/2165689998 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f875c0a3c40 0x7f875c0a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8760006010 tx=0x7f8760009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:52.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.532+0000 7f877482c700 1 -- 172.21.15.19:0/2165689998 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8758020020 con 0x7f8770103cb0 2024-04-25T12:06:52.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.789+0000 7f877582e700 1 -- 172.21.15.19:0/2165689998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8770063650 con 0x7f8770103cb0 2024-04-25T12:06:52.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.789+0000 7f877482c700 1 -- 172.21.15.19:0/2165689998 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3098 (secure 0 0 0) 0x7f875809cbf0 con 0x7f8770103cb0 2024-04-25T12:06:52.788 INFO:teuthology.orchestra.run.smithi019.stdout:e26 2024-04-25T12:06:52.788 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T12:06:52.788 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T12:06:52.788 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-04-25T12:06:52.788 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:52.788 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-04-25T12:06:52.788 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-04-25T12:06:52.788 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 26 2024-04-25T12:06:52.788 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T12:06:52.788 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-04-25T12:05:04.008679+0000 2024-04-25T12:06:52.788 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-04-25T12:05:11.186222+0000 2024-04-25T12:06:52.788 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-04-25T12:06:52.789 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-04-25T12:06:52.789 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 300 2024-04-25T12:06:52.789 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-04-25T12:06:52.789 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-04-25T12:06:52.789 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-04-25T12:06:52.789 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-04-25T12:06:52.789 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-04-25T12:06:52.789 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 0 2024-04-25T12:06:52.789 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T12:06:52.789 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 5 2024-04-25T12:06:52.789 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1,2,3,4 2024-04-25T12:06:52.789 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=24514,1=24500,2=24496,3=24484,4=14655} 2024-04-25T12:06:52.789 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-04-25T12:06:52.790 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-04-25T12:06:52.790 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-04-25T12:06:52.790 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-04-25T12:06:52.790 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-04-25T12:06:52.790 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-04-25T12:06:52.790 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-04-25T12:06:52.790 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-04-25T12:06:52.790 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 0 2024-04-25T12:06:52.790 INFO:teuthology.orchestra.run.smithi019.stdout:qdb_cluster leader: 24514 members: 24500,24514,24496,24484,14655 2024-04-25T12:06:52.790 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.i{0:24514} state up:active seq 5 addr [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T12:06:52.790 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.h{0:24470} state up:standby-replay seq 1 addr [v2:172.21.15.150:6836/3918801450,v1:172.21.15.150:6837/3918801450] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T12:06:52.790 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b{1:24500} state up:active seq 4 addr [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T12:06:52.790 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.e{1:24440} state up:standby-replay seq 1 addr [v2:172.21.15.150:6834/1405540140,v1:172.21.15.150:6835/1405540140] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T12:06:52.790 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c{2:24496} state up:active seq 8 addr [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T12:06:52.791 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.l{2:24427} state up:standby-replay seq 1 addr [v2:172.21.15.169:6832/2217300694,v1:172.21.15.169:6833/2217300694] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T12:06:52.791 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.f{3:24484} state up:active seq 9 addr [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T12:06:52.791 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.k{3:24404} state up:standby-replay seq 1 addr [v2:172.21.15.150:6832/4176941259,v1:172.21.15.150:6833/4176941259] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T12:06:52.791 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a{4:14655} state up:active seq 5 addr [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T12:06:52.791 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.g{4:14643} state up:standby-replay seq 1 addr [v2:172.21.15.19:6838/1694897630,v1:172.21.15.19:6839/1694897630] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T12:06:52.791 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:52.791 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:52.791 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-04-25T12:06:52.791 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:52.791 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.j{-1:14559} state up:standby seq 1 addr [v2:172.21.15.19:6834/3292992712,v1:172.21.15.19:6835/3292992712] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T12:06:52.791 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d{-1:14577} state up:standby seq 1 addr [v2:172.21.15.19:6836/3684868666,v1:172.21.15.19:6837/3684868666] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T12:06:52.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.791+0000 7f877582e700 1 -- 172.21.15.19:0/2165689998 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f875c0a3c40 msgr2=0x7f875c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:52.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.791+0000 7f877582e700 1 --2- 172.21.15.19:0/2165689998 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f875c0a3c40 0x7f875c0a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8760006010 tx=0x7f8760009f10 comp rx=0 tx=0).stop 2024-04-25T12:06:52.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.791+0000 7f877582e700 1 -- 172.21.15.19:0/2165689998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8770103cb0 msgr2=0x7f8770199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:52.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.791+0000 7f877582e700 1 --2- 172.21.15.19:0/2165689998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8770103cb0 0x7f8770199240 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f875800d980 tx=0x7f875800dc90 comp rx=0 tx=0).stop 2024-04-25T12:06:52.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.791+0000 7f877582e700 1 -- 172.21.15.19:0/2165689998 shutdown_connections 2024-04-25T12:06:52.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.791+0000 7f877582e700 1 --2- 172.21.15.19:0/2165689998 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f875c0a3c40 0x7f875c0a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:52.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.791+0000 7f877582e700 1 --2- 172.21.15.19:0/2165689998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8770104650 0x7f87701a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:52.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.791+0000 7f877582e700 1 --2- 172.21.15.19:0/2165689998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8770102ab0 0x7f8770198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:52.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.791+0000 7f877582e700 1 --2- 172.21.15.19:0/2165689998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8770103cb0 0x7f8770199240 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:52.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.792+0000 7f877582e700 1 -- 172.21.15.19:0/2165689998 >> 172.21.15.19:0/2165689998 conn(0x7f87700fe050 msgr2=0x7f8770106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:52.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.792+0000 7f877582e700 1 -- 172.21.15.19:0/2165689998 shutdown_connections 2024-04-25T12:06:52.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:52.792+0000 7f877582e700 1 -- 172.21.15.19:0/2165689998 wait complete. 2024-04-25T12:06:52.793 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-25T12:06:53.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:52 smithi019 ceph-mon[29213]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T12:06:53.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:52 smithi169 ceph-mon[32710]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T12:06:53.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:52 smithi150 ceph-mon[32250]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T12:06:53.607 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T12:06:53.863 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:06:53.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:53 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2165689998' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T12:06:54.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:53 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2165689998' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T12:06:54.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:53 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2165689998' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T12:06:55.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:54 smithi019 ceph-mon[29213]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T12:06:55.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:54 smithi169 ceph-mon[32710]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T12:06:55.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:54 smithi150 ceph-mon[32250]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T12:06:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.773+0000 7f90b9dc5700 1 -- 172.21.15.19:0/2190704456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b4103cc0 msgr2=0x7f90b4104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.773+0000 7f90b9dc5700 1 --2- 172.21.15.19:0/2190704456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b4103cc0 0x7f90b4104120 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f90a80099c0 tx=0x7f90a8009cd0 comp rx=0 tx=0).stop 2024-04-25T12:06:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.773+0000 7f90b9dc5700 1 -- 172.21.15.19:0/2190704456 shutdown_connections 2024-04-25T12:06:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.773+0000 7f90b9dc5700 1 --2- 172.21.15.19:0/2190704456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b4104660 0x7f90b410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.773+0000 7f90b9dc5700 1 --2- 172.21.15.19:0/2190704456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b4102ac0 0x7f90b4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.773+0000 7f90b9dc5700 1 --2- 172.21.15.19:0/2190704456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b4103cc0 0x7f90b4104120 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.773+0000 7f90b9dc5700 1 -- 172.21.15.19:0/2190704456 >> 172.21.15.19:0/2190704456 conn(0x7f90b40fe060 msgr2=0x7f90b41004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.773+0000 7f90b9dc5700 1 -- 172.21.15.19:0/2190704456 shutdown_connections 2024-04-25T12:06:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.773+0000 7f90b9dc5700 1 -- 172.21.15.19:0/2190704456 wait complete. 2024-04-25T12:06:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.774+0000 7f90b9dc5700 1 Processor -- start 2024-04-25T12:06:55.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.774+0000 7f90b9dc5700 1 -- start start 2024-04-25T12:06:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.774+0000 7f90b9dc5700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b4102ac0 0x7f90b40767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.774+0000 7f90b9dc5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b4103cc0 0x7f90b4074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.774+0000 7f90b9dc5700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b4104660 0x7f90b4075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b9dc5700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f90b410e790 con 0x7f90b4104660 2024-04-25T12:06:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b9dc5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f90b410e610 con 0x7f90b4103cc0 2024-04-25T12:06:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b9dc5700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f90b410e910 con 0x7f90b4102ac0 2024-04-25T12:06:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b4104660 0x7f90b4075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b3fff700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b4104660 0x7f90b4075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55164/0 (socket says 172.21.15.19:55164) 2024-04-25T12:06:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b3fff700 1 -- 172.21.15.19:0/1619034679 learned_addr learned my addr 172.21.15.19:0/1619034679 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b37fe700 1 --2- 172.21.15.19:0/1619034679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b4102ac0 0x7f90b40767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b2ffd700 1 --2- 172.21.15.19:0/1619034679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b4103cc0 0x7f90b4074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b3fff700 1 -- 172.21.15.19:0/1619034679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b4102ac0 msgr2=0x7f90b40767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b3fff700 1 --2- 172.21.15.19:0/1619034679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b4102ac0 0x7f90b40767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b3fff700 1 -- 172.21.15.19:0/1619034679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b4103cc0 msgr2=0x7f90b4074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:55.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b3fff700 1 --2- 172.21.15.19:0/1619034679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b4103cc0 0x7f90b4074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:55.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b3fff700 1 -- 172.21.15.19:0/1619034679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b4075c20 con 0x7f90b4104660 2024-04-25T12:06:55.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b3fff700 1 --2- 172.21.15.19:0/1619034679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b4104660 0x7f90b4075370 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00b970 tx=0x7f90ac00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:55.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b0ff9700 1 -- 172.21.15.19:0/1619034679 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac019070 con 0x7f90b4104660 2024-04-25T12:06:55.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b0ff9700 1 -- 172.21.15.19:0/1619034679 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f90ac00ce00 con 0x7f90b4104660 2024-04-25T12:06:55.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.775+0000 7f90b9dc5700 1 -- 172.21.15.19:0/1619034679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b41a2f10 con 0x7f90b4104660 2024-04-25T12:06:55.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.776+0000 7f90b0ff9700 1 -- 172.21.15.19:0/1619034679 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac007370 con 0x7f90b4104660 2024-04-25T12:06:55.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.776+0000 7f90b9dc5700 1 -- 172.21.15.19:0/1619034679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b41a33f0 con 0x7f90b4104660 2024-04-25T12:06:55.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.776+0000 7f90b9dc5700 1 -- 172.21.15.19:0/1619034679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b4069730 con 0x7f90b4104660 2024-04-25T12:06:55.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.778+0000 7f90b0ff9700 1 -- 172.21.15.19:0/1619034679 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f90ac01c070 con 0x7f90b4104660 2024-04-25T12:06:55.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.778+0000 7f90b0ff9700 1 --2- 172.21.15.19:0/1619034679 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f909c0a3cc0 0x7f909c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:55.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.778+0000 7f90b37fe700 1 --2- 172.21.15.19:0/1619034679 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f909c0a3cc0 0x7f909c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:55.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.778+0000 7f90b0ff9700 1 -- 172.21.15.19:0/1619034679 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f90ac0cfba0 con 0x7f90b4104660 2024-04-25T12:06:55.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.779+0000 7f90b37fe700 1 --2- 172.21.15.19:0/1619034679 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f909c0a3cc0 0x7f909c0a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f90b4103b20 tx=0x7f90a4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:55.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.782+0000 7f90b0ff9700 1 -- 172.21.15.19:0/1619034679 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90ac09c560 con 0x7f90b4104660 2024-04-25T12:06:55.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.991+0000 7f90b9dc5700 1 -- 172.21.15.19:0/1619034679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f90b402e050 con 0x7f90b4104660 2024-04-25T12:06:55.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.992+0000 7f90b0ff9700 1 -- 172.21.15.19:0/1619034679 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5328 (secure 0 0 0) 0x7f90ac09be60 con 0x7f90b4104660 2024-04-25T12:06:55.990 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 87 2024-04-25T12:06:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:fsid be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T12:06:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-04-25T11:57:00.082417+0000 2024-04-25T12:06:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-04-25T12:05:55.975412+0000 2024-04-25T12:06:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T12:06:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:crush_version 26 2024-04-25T12:06:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:full_ratio 0.95 2024-04-25T12:06:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:backfillfull_ratio 0.9 2024-04-25T12:06:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:nearfull_ratio 0.85 2024-04-25T12:06:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:require_min_compat_client luminous 2024-04-25T12:06:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:min_compat_client luminous 2024-04-25T12:06:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:require_osd_release squid 2024-04-25T12:06:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:stretch_mode_enabled false 2024-04-25T12:06:55.991 INFO:teuthology.orchestra.run.smithi019.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T12:06:55.992 INFO:teuthology.orchestra.run.smithi019.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T12:06:55.992 INFO:teuthology.orchestra.run.smithi019.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T12:06:55.992 INFO:teuthology.orchestra.run.smithi019.stdout:max_osd 12 2024-04-25T12:06:55.992 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6802/86476755,v1:172.21.15.19:6803/86476755] [v2:172.21.15.19:6804/86476755,v1:172.21.15.19:6805/86476755] exists,up 73d2b9f6-57f7-4008-9269-75c46243c8e0 2024-04-25T12:06:55.992 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/1593189601,v1:172.21.15.150:6801/1593189601] [v2:172.21.15.150:6802/1593189601,v1:172.21.15.150:6803/1593189601] exists,up 9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0 2024-04-25T12:06:55.992 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6800/3641724528,v1:172.21.15.169:6801/3641724528] [v2:172.21.15.169:6802/3641724528,v1:172.21.15.169:6803/3641724528] exists,up 3a12732f-3dd6-4d97-b62a-2bd66a797b1c 2024-04-25T12:06:55.992 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6810/3285522913,v1:172.21.15.19:6811/3285522913] [v2:172.21.15.19:6812/3285522913,v1:172.21.15.19:6813/3285522913] exists,up 518328cd-b7de-4a49-9045-2ebf6b0eb4ab 2024-04-25T12:06:55.992 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/2162522887,v1:172.21.15.150:6809/2162522887] [v2:172.21.15.150:6810/2162522887,v1:172.21.15.150:6811/2162522887] exists,up 14ddc9b5-1675-428c-b759-44c255d90bac 2024-04-25T12:06:55.992 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6808/502865081,v1:172.21.15.169:6809/502865081] [v2:172.21.15.169:6810/502865081,v1:172.21.15.169:6811/502865081] exists,up 99b7ca08-7d93-405d-9f28-9a98b7b3f669 2024-04-25T12:06:55.992 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 up in weight 1 up_from 46 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6818/814529084,v1:172.21.15.19:6819/814529084] [v2:172.21.15.19:6820/814529084,v1:172.21.15.19:6821/814529084] exists,up fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac 2024-04-25T12:06:55.992 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 up in weight 1 up_from 51 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/4020549658,v1:172.21.15.150:6817/4020549658] [v2:172.21.15.150:6818/4020549658,v1:172.21.15.150:6819/4020549658] exists,up e19b7274-6545-43ee-891a-30f25c2c2e41 2024-04-25T12:06:55.993 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6816/3896931254,v1:172.21.15.169:6817/3896931254] [v2:172.21.15.169:6818/3896931254,v1:172.21.15.169:6819/3896931254] exists,up 8175ec26-2033-4312-8778-2d2e8512e4cf 2024-04-25T12:06:55.993 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6826/3504107664,v1:172.21.15.19:6827/3504107664] [v2:172.21.15.19:6828/3504107664,v1:172.21.15.19:6829/3504107664] exists,up 2941192e-f2a5-447c-8185-61f976b616db 2024-04-25T12:06:55.993 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 up in weight 1 up_from 72 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/2984224024,v1:172.21.15.150:6825/2984224024] [v2:172.21.15.150:6826/2984224024,v1:172.21.15.150:6827/2984224024] exists,up e97514b7-bafc-4e07-8afb-3d33ab586ef9 2024-04-25T12:06:55.993 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.169:6824/2387365529,v1:172.21.15.169:6825/2387365529] [v2:172.21.15.169:6826/2387365529,v1:172.21.15.169:6827/2387365529] exists,up 98d4c806-cf93-41d4-9b62-9febe86e9372 2024-04-25T12:06:55.993 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.e [2,4] 2024-04-25T12:06:55.993 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.16 [2,4] 2024-04-25T12:06:55.993 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.1b [5,11] 2024-04-25T12:06:55.993 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.1d [8,11] 2024-04-25T12:06:55.993 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.1f [8,1] 2024-04-25T12:06:55.993 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.6 [4,6] 2024-04-25T12:06:55.993 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.12 [3,9] 2024-04-25T12:06:55.993 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.24 [4,9] 2024-04-25T12:06:55.993 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.36 [4,9] 2024-04-25T12:06:55.994 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.39 [5,8] 2024-04-25T12:06:55.994 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/37557166 expires 2024-04-26T11:57:54.811890+0000 2024-04-25T12:06:55.994 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/963006875 expires 2024-04-26T11:57:54.811890+0000 2024-04-25T12:06:55.994 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/963006875 expires 2024-04-26T11:57:54.811890+0000 2024-04-25T12:06:55.994 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/556620573 expires 2024-04-26T11:57:27.561732+0000 2024-04-25T12:06:55.994 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/556620573 expires 2024-04-26T11:57:27.561732+0000 2024-04-25T12:06:55.994 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1079165232 expires 2024-04-26T11:57:54.811890+0000 2024-04-25T12:06:55.994 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/4104025927 expires 2024-04-26T11:57:27.561732+0000 2024-04-25T12:06:55.994 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/2968863069 expires 2024-04-26T11:57:54.811890+0000 2024-04-25T12:06:55.994 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3693313603 expires 2024-04-26T11:57:27.561732+0000 2024-04-25T12:06:55.994 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1919062753 expires 2024-04-26T11:57:27.561732+0000 2024-04-25T12:06:55.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.994+0000 7f90b9dc5700 1 -- 172.21.15.19:0/1619034679 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f909c0a3cc0 msgr2=0x7f909c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:55.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.994+0000 7f90b9dc5700 1 --2- 172.21.15.19:0/1619034679 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f909c0a3cc0 0x7f909c0a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f90b4103b20 tx=0x7f90a4009190 comp rx=0 tx=0).stop 2024-04-25T12:06:55.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.994+0000 7f90b9dc5700 1 -- 172.21.15.19:0/1619034679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b4104660 msgr2=0x7f90b4075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:55.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.994+0000 7f90b9dc5700 1 --2- 172.21.15.19:0/1619034679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b4104660 0x7f90b4075370 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00b970 tx=0x7f90ac00bc80 comp rx=0 tx=0).stop 2024-04-25T12:06:55.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.995+0000 7f90b9dc5700 1 -- 172.21.15.19:0/1619034679 shutdown_connections 2024-04-25T12:06:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.995+0000 7f90b9dc5700 1 --2- 172.21.15.19:0/1619034679 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f909c0a3cc0 0x7f909c0a6180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.995+0000 7f90b9dc5700 1 --2- 172.21.15.19:0/1619034679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90b4104660 0x7f90b4075370 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.995+0000 7f90b9dc5700 1 --2- 172.21.15.19:0/1619034679 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90b4102ac0 0x7f90b40767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.995+0000 7f90b9dc5700 1 --2- 172.21.15.19:0/1619034679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b4103cc0 0x7f90b4074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.995+0000 7f90b9dc5700 1 -- 172.21.15.19:0/1619034679 >> 172.21.15.19:0/1619034679 conn(0x7f90b40fe060 msgr2=0x7f90b4106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.995+0000 7f90b9dc5700 1 -- 172.21.15.19:0/1619034679 shutdown_connections 2024-04-25T12:06:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:55.995+0000 7f90b9dc5700 1 -- 172.21.15.19:0/1619034679 wait complete. 2024-04-25T12:06:56.772 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:56 smithi019 ceph-mon[29213]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:06:56.773 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:56 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1619034679' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T12:06:56.808 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T12:06:56.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:06:56.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.935+0000 7f8027223640 1 -- 172.21.15.19:0/2723537056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020100e90 msgr2=0x7f80201012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:56.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.935+0000 7f8027223640 1 --2- 172.21.15.19:0/2723537056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020100e90 0x7f80201012b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8010009900 tx=0x7f801002a410 comp rx=0 tx=0).stop 2024-04-25T12:06:56.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.935+0000 7f8027223640 1 -- 172.21.15.19:0/2723537056 shutdown_connections 2024-04-25T12:06:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.935+0000 7f8027223640 1 --2- 172.21.15.19:0/2723537056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80201029a0 0x7f80201092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.935+0000 7f8027223640 1 --2- 172.21.15.19:0/2723537056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8020101fe0 0x7f8020102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.935+0000 7f8027223640 1 --2- 172.21.15.19:0/2723537056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020100e90 0x7f80201012b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.935+0000 7f8027223640 1 -- 172.21.15.19:0/2723537056 >> 172.21.15.19:0/2723537056 conn(0x7f80200fc400 msgr2=0x7f80200fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.935+0000 7f8027223640 1 -- 172.21.15.19:0/2723537056 shutdown_connections 2024-04-25T12:06:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.935+0000 7f8027223640 1 -- 172.21.15.19:0/2723537056 wait complete. 2024-04-25T12:06:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f8027223640 1 Processor -- start 2024-04-25T12:06:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f8027223640 1 -- start start 2024-04-25T12:06:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f8027223640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8020100e90 0x7f8020069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f8027223640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020101fe0 0x7f802019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f8027223640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80201029a0 0x7f802019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f8027223640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f802010cb20 con 0x7f8020101fe0 2024-04-25T12:06:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f8027223640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f802010c9a0 con 0x7f80201029a0 2024-04-25T12:06:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f8027223640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f802010cca0 con 0x7f8020100e90 2024-04-25T12:06:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f801ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020101fe0 0x7f802019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f801ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020101fe0 0x7f802019d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55188/0 (socket says 172.21.15.19:55188) 2024-04-25T12:06:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f801ffff640 1 -- 172.21.15.19:0/853267656 learned_addr learned my addr 172.21.15.19:0/853267656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f80257c0640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80201029a0 0x7f802019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f801ffff640 1 -- 172.21.15.19:0/853267656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8020100e90 msgr2=0x7f8020069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f8024fbf640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8020100e90 0x7f8020069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f801ffff640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8020100e90 0x7f8020069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f801ffff640 1 -- 172.21.15.19:0/853267656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80201029a0 msgr2=0x7f802019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f801ffff640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80201029a0 0x7f802019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f801ffff640 1 -- 172.21.15.19:0/853267656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802019ff50 con 0x7f8020101fe0 2024-04-25T12:06:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f80257c0640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80201029a0 0x7f802019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:06:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.936+0000 7f801ffff640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020101fe0 0x7f802019d020 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f801400d950 tx=0x7f801400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.937+0000 7f801dffb640 1 -- 172.21.15.19:0/853267656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014017070 con 0x7f8020101fe0 2024-04-25T12:06:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.937+0000 7f801dffb640 1 -- 172.21.15.19:0/853267656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8014009c30 con 0x7f8020101fe0 2024-04-25T12:06:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.937+0000 7f8027223640 1 -- 172.21.15.19:0/853267656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80201a01e0 con 0x7f8020101fe0 2024-04-25T12:06:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.937+0000 7f801dffb640 1 -- 172.21.15.19:0/853267656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801400f620 con 0x7f8020101fe0 2024-04-25T12:06:56.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.937+0000 7f8027223640 1 -- 172.21.15.19:0/853267656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80201a0730 con 0x7f8020101fe0 2024-04-25T12:06:56.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.938+0000 7f8027223640 1 -- 172.21.15.19:0/853267656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80201081a0 con 0x7f8020101fe0 2024-04-25T12:06:56.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.939+0000 7f801dffb640 1 -- 172.21.15.19:0/853267656 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f801401a070 con 0x7f8020101fe0 2024-04-25T12:06:56.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.940+0000 7f801dffb640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f80080a3c30 0x7f80080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:56.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.940+0000 7f8024fbf640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f80080a3c30 0x7f80080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:56.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.940+0000 7f801dffb640 1 -- 172.21.15.19:0/853267656 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f80140cff20 con 0x7f8020101fe0 2024-04-25T12:06:56.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.940+0000 7f8024fbf640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f80080a3c30 0x7f80080a60f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8010031570 tx=0x7f8010000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:56.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:56.944+0000 7f801dffb640 1 -- 172.21.15.19:0/853267656 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8014009de0 con 0x7f8020101fe0 2024-04-25T12:06:57.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.133+0000 7f8027223640 1 -- 172.21.15.19:0/853267656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8020062b80 con 0x7f8020101fe0 2024-04-25T12:06:57.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.135+0000 7f801dffb640 1 -- 172.21.15.19:0/853267656 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22731 (secure 0 0 0) 0x7f801409c880 con 0x7f8020101fe0 2024-04-25T12:06:57.134 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:57.134 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":87,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:05:55.975412+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:05:01.658354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:05:02.969078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:06:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.137+0000 7f8027223640 1 -- 172.21.15.19:0/853267656 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f80080a3c30 msgr2=0x7f80080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.137+0000 7f8027223640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f80080a3c30 0x7f80080a60f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8010031570 tx=0x7f8010000bc0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.137+0000 7f8027223640 1 -- 172.21.15.19:0/853267656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020101fe0 msgr2=0x7f802019d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.137+0000 7f8027223640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020101fe0 0x7f802019d020 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f801400d950 tx=0x7f801400de10 comp rx=0 tx=0).stop 2024-04-25T12:06:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.137+0000 7f8027223640 1 -- 172.21.15.19:0/853267656 shutdown_connections 2024-04-25T12:06:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.137+0000 7f8027223640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f80080a3c30 0x7f80080a60f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.137+0000 7f8027223640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80201029a0 0x7f802019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.137+0000 7f8027223640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8020101fe0 0x7f802019d020 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.137+0000 7f8027223640 1 --2- 172.21.15.19:0/853267656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8020100e90 0x7f8020069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.137+0000 7f8027223640 1 -- 172.21.15.19:0/853267656 >> 172.21.15.19:0/853267656 conn(0x7f80200fc400 msgr2=0x7f8020105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.137+0000 7f8027223640 1 -- 172.21.15.19:0/853267656 shutdown_connections 2024-04-25T12:06:57.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.137+0000 7f8027223640 1 -- 172.21.15.19:0/853267656 wait complete. 2024-04-25T12:06:57.147 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:00:20.898809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:05:01.658354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:05:02.969078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T12:06:57.148 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:06:57.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:56 smithi169 ceph-mon[32710]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:06:57.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:56 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1619034679' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T12:06:57.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:56 smithi150 ceph-mon[32250]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:06:57.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:56 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1619034679' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T12:06:57.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.255+0000 7f197587b640 1 -- 172.21.15.19:0/2493771288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1970107d30 msgr2=0x7f197010a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.255+0000 7f197587b640 1 --2- 172.21.15.19:0/2493771288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1970107d30 0x7f197010a120 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1968009900 tx=0x7f196802a410 comp rx=0 tx=0).stop 2024-04-25T12:06:57.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.255+0000 7f197587b640 1 -- 172.21.15.19:0/2493771288 shutdown_connections 2024-04-25T12:06:57.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.255+0000 7f197587b640 1 --2- 172.21.15.19:0/2493771288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1970107d30 0x7f197010a120 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.255+0000 7f197587b640 1 --2- 172.21.15.19:0/2493771288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1970105400 0x7f19701077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.255+0000 7f197587b640 1 --2- 172.21.15.19:0/2493771288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1970102ad0 0x7f1970104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.255+0000 7f197587b640 1 -- 172.21.15.19:0/2493771288 >> 172.21.15.19:0/2493771288 conn(0x7f19700fc400 msgr2=0x7f19700fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:57.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.255+0000 7f197587b640 1 -- 172.21.15.19:0/2493771288 shutdown_connections 2024-04-25T12:06:57.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.255+0000 7f197587b640 1 -- 172.21.15.19:0/2493771288 wait complete. 2024-04-25T12:06:57.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.256+0000 7f197587b640 1 Processor -- start 2024-04-25T12:06:57.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.256+0000 7f197587b640 1 -- start start 2024-04-25T12:06:57.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.256+0000 7f197587b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1970102ad0 0x7f1970198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:57.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.256+0000 7f197587b640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1970105400 0x7f19701987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:57.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.256+0000 7f197587b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1970107d30 0x7f197019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:57.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.256+0000 7f197587b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f197010dc50 con 0x7f1970102ad0 2024-04-25T12:06:57.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.256+0000 7f197587b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f197010dad0 con 0x7f1970107d30 2024-04-25T12:06:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.256+0000 7f197587b640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f197010ddd0 con 0x7f1970105400 2024-04-25T12:06:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.256+0000 7f196effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1970102ad0 0x7f1970198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.256+0000 7f196effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1970102ad0 0x7f1970198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55210/0 (socket says 172.21.15.19:55210) 2024-04-25T12:06:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.256+0000 7f196effd640 1 -- 172.21.15.19:0/4281888646 learned_addr learned my addr 172.21.15.19:0/4281888646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f196f7fe640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1970107d30 0x7f197019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f196e7fc640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1970105400 0x7f19701987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f196effd640 1 -- 172.21.15.19:0/4281888646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1970105400 msgr2=0x7f19701987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f196effd640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1970105400 0x7f19701987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f196effd640 1 -- 172.21.15.19:0/4281888646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1970107d30 msgr2=0x7f197019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f196effd640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1970107d30 0x7f197019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f196effd640 1 -- 172.21.15.19:0/4281888646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197019fe90 con 0x7f1970102ad0 2024-04-25T12:06:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f196e7fc640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1970105400 0x7f19701987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:06:57.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f196f7fe640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1970107d30 0x7f197019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:57.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f196effd640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1970102ad0 0x7f1970198270 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f196000cce0 tx=0x7f196000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:57.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f1974879640 1 -- 172.21.15.19:0/4281888646 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960004b20 con 0x7f1970102ad0 2024-04-25T12:06:57.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f1974879640 1 -- 172.21.15.19:0/4281888646 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f1960004cc0 con 0x7f1970102ad0 2024-04-25T12:06:57.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f197587b640 1 -- 172.21.15.19:0/4281888646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19701a0180 con 0x7f1970102ad0 2024-04-25T12:06:57.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f1974879640 1 -- 172.21.15.19:0/4281888646 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196000f760 con 0x7f1970102ad0 2024-04-25T12:06:57.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.257+0000 7f197587b640 1 -- 172.21.15.19:0/4281888646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19701a06d0 con 0x7f1970102ad0 2024-04-25T12:06:57.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.258+0000 7f197587b640 1 -- 172.21.15.19:0/4281888646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19700fdef0 con 0x7f1970102ad0 2024-04-25T12:06:57.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.259+0000 7f1974879640 1 -- 172.21.15.19:0/4281888646 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1960005580 con 0x7f1970102ad0 2024-04-25T12:06:57.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.260+0000 7f1974879640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f19580a3bb0 0x7f19580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:57.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.260+0000 7f196e7fc640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f19580a3bb0 0x7f19580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:57.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.260+0000 7f1974879640 1 -- 172.21.15.19:0/4281888646 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f19600d0520 con 0x7f1970102ad0 2024-04-25T12:06:57.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.260+0000 7f196e7fc640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f19580a3bb0 0x7f19580a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1970199780 tx=0x7f1964005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:57.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.264+0000 7f1974879640 1 -- 172.21.15.19:0/4281888646 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1960014030 con 0x7f1970102ad0 2024-04-25T12:06:57.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.449+0000 7f197587b640 1 -- 172.21.15.19:0/4281888646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1970062b80 con 0x7f1970102ad0 2024-04-25T12:06:57.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.449+0000 7f1974879640 1 -- 172.21.15.19:0/4281888646 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f196009cee0 con 0x7f1970102ad0 2024-04-25T12:06:57.447 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T12:06:57.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.451+0000 7f197587b640 1 -- 172.21.15.19:0/4281888646 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f19580a3bb0 msgr2=0x7f19580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.451+0000 7f197587b640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f19580a3bb0 0x7f19580a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1970199780 tx=0x7f1964005d30 comp rx=0 tx=0).stop 2024-04-25T12:06:57.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.451+0000 7f197587b640 1 -- 172.21.15.19:0/4281888646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1970102ad0 msgr2=0x7f1970198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.451+0000 7f197587b640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1970102ad0 0x7f1970198270 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f196000cce0 tx=0x7f196000a620 comp rx=0 tx=0).stop 2024-04-25T12:06:57.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.451+0000 7f197587b640 1 -- 172.21.15.19:0/4281888646 shutdown_connections 2024-04-25T12:06:57.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.451+0000 7f197587b640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f19580a3bb0 0x7f19580a6070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.451+0000 7f197587b640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1970107d30 0x7f197019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.451+0000 7f197587b640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1970105400 0x7f19701987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.451+0000 7f197587b640 1 --2- 172.21.15.19:0/4281888646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1970102ad0 0x7f1970198270 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.451+0000 7f197587b640 1 -- 172.21.15.19:0/4281888646 >> 172.21.15.19:0/4281888646 conn(0x7f19700fc400 msgr2=0x7f1970103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:57.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.451+0000 7f197587b640 1 -- 172.21.15.19:0/4281888646 shutdown_connections 2024-04-25T12:06:57.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.451+0000 7f197587b640 1 -- 172.21.15.19:0/4281888646 wait complete. 2024-04-25T12:06:57.461 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T12:06:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.567+0000 7f80854e1640 1 -- 172.21.15.19:0/3020231211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8080101fc0 msgr2=0x7f8080102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.567+0000 7f80854e1640 1 --2- 172.21.15.19:0/3020231211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8080101fc0 0x7f8080102440 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8068009900 tx=0x7f806802a410 comp rx=0 tx=0).stop 2024-04-25T12:06:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.567+0000 7f80854e1640 1 -- 172.21.15.19:0/3020231211 shutdown_connections 2024-04-25T12:06:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.567+0000 7f80854e1640 1 --2- 172.21.15.19:0/3020231211 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8080102980 0x7f80801092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.567+0000 7f80854e1640 1 --2- 172.21.15.19:0/3020231211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8080101fc0 0x7f8080102440 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.567+0000 7f80854e1640 1 --2- 172.21.15.19:0/3020231211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8080100e70 0x7f8080101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.567+0000 7f80854e1640 1 -- 172.21.15.19:0/3020231211 >> 172.21.15.19:0/3020231211 conn(0x7f80800fc3e0 msgr2=0x7f80800fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.567+0000 7f80854e1640 1 -- 172.21.15.19:0/3020231211 shutdown_connections 2024-04-25T12:06:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.567+0000 7f80854e1640 1 -- 172.21.15.19:0/3020231211 wait complete. 2024-04-25T12:06:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.568+0000 7f80854e1640 1 Processor -- start 2024-04-25T12:06:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.568+0000 7f80854e1640 1 -- start start 2024-04-25T12:06:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.568+0000 7f80854e1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8080100e70 0x7f8080198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.568+0000 7f80854e1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8080101fc0 0x7f8080198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.568+0000 7f80854e1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8080102980 0x7f808019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.568+0000 7f80854e1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f808010dc20 con 0x7f8080102980 2024-04-25T12:06:57.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.568+0000 7f80854e1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f808010daa0 con 0x7f8080101fc0 2024-04-25T12:06:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.568+0000 7f80854e1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f808010dda0 con 0x7f8080100e70 2024-04-25T12:06:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.568+0000 7f807f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8080102980 0x7f808019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.568+0000 7f807f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8080102980 0x7f808019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55226/0 (socket says 172.21.15.19:55226) 2024-04-25T12:06:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.568+0000 7f807f7fe640 1 -- 172.21.15.19:0/1626758495 learned_addr learned my addr 172.21.15.19:0/1626758495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.568+0000 7f807effd640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8080100e70 0x7f8080198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.569+0000 7f807e7fc640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8080101fc0 0x7f8080198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.569+0000 7f807f7fe640 1 -- 172.21.15.19:0/1626758495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8080100e70 msgr2=0x7f8080198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.569+0000 7f807f7fe640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8080100e70 0x7f8080198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.569+0000 7f807f7fe640 1 -- 172.21.15.19:0/1626758495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8080101fc0 msgr2=0x7f8080198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.569+0000 7f807f7fe640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8080101fc0 0x7f8080198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.569+0000 7f807f7fe640 1 -- 172.21.15.19:0/1626758495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808019fe60 con 0x7f8080102980 2024-04-25T12:06:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.569+0000 7f807e7fc640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8080101fc0 0x7f8080198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:06:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.569+0000 7f807f7fe640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8080102980 0x7f808019f7f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f807800c970 tx=0x7f807800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.569+0000 7f8077fff640 1 -- 172.21.15.19:0/1626758495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078007bb0 con 0x7f8080102980 2024-04-25T12:06:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.569+0000 7f8077fff640 1 -- 172.21.15.19:0/1626758495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8078007d50 con 0x7f8080102980 2024-04-25T12:06:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.569+0000 7f80854e1640 1 -- 172.21.15.19:0/1626758495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80801a0150 con 0x7f8080102980 2024-04-25T12:06:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.569+0000 7f8077fff640 1 -- 172.21.15.19:0/1626758495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807800f450 con 0x7f8080102980 2024-04-25T12:06:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.569+0000 7f80854e1640 1 -- 172.21.15.19:0/1626758495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80801a06a0 con 0x7f8080102980 2024-04-25T12:06:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.570+0000 7f80854e1640 1 -- 172.21.15.19:0/1626758495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8080108180 con 0x7f8080102980 2024-04-25T12:06:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.571+0000 7f8077fff640 1 -- 172.21.15.19:0/1626758495 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8078004f20 con 0x7f8080102980 2024-04-25T12:06:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.572+0000 7f8077fff640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f806c0a3bb0 0x7f806c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.572+0000 7f807effd640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f806c0a3bb0 0x7f806c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.572+0000 7f8077fff640 1 -- 172.21.15.19:0/1626758495 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f80780d0490 con 0x7f8080102980 2024-04-25T12:06:57.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.572+0000 7f807effd640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f806c0a3bb0 0x7f806c0a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8070007ec0 tx=0x7f80700075b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.575+0000 7f8077fff640 1 -- 172.21.15.19:0/1626758495 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8078014030 con 0x7f8080102980 2024-04-25T12:06:57.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.764+0000 7f80854e1640 1 -- 172.21.15.19:0/1626758495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f808002d5c0 con 0x7f8080102980 2024-04-25T12:06:57.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.764+0000 7f8077fff640 1 -- 172.21.15.19:0/1626758495 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f807809ce50 con 0x7f8080102980 2024-04-25T12:06:57.762 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T12:06:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.766+0000 7f80854e1640 1 -- 172.21.15.19:0/1626758495 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f806c0a3bb0 msgr2=0x7f806c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.766+0000 7f80854e1640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f806c0a3bb0 0x7f806c0a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8070007ec0 tx=0x7f80700075b0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.766+0000 7f80854e1640 1 -- 172.21.15.19:0/1626758495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8080102980 msgr2=0x7f808019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.766+0000 7f80854e1640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8080102980 0x7f808019f7f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f807800c970 tx=0x7f807800ce30 comp rx=0 tx=0).stop 2024-04-25T12:06:57.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.766+0000 7f80854e1640 1 -- 172.21.15.19:0/1626758495 shutdown_connections 2024-04-25T12:06:57.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.766+0000 7f80854e1640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f806c0a3bb0 0x7f806c0a6070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.766+0000 7f80854e1640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8080102980 0x7f808019f7f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.766+0000 7f80854e1640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8080101fc0 0x7f8080198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.766+0000 7f80854e1640 1 --2- 172.21.15.19:0/1626758495 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8080100e70 0x7f8080198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.766+0000 7f80854e1640 1 -- 172.21.15.19:0/1626758495 >> 172.21.15.19:0/1626758495 conn(0x7f80800fc3e0 msgr2=0x7f8080105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:57.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.766+0000 7f80854e1640 1 -- 172.21.15.19:0/1626758495 shutdown_connections 2024-04-25T12:06:57.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.766+0000 7f80854e1640 1 -- 172.21.15.19:0/1626758495 wait complete. 2024-04-25T12:06:57.775 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T12:06:57.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.882+0000 7f09975a8640 1 -- 172.21.15.19:0/799630445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0990105360 msgr2=0x7f0990107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.882+0000 7f09975a8640 1 --2- 172.21.15.19:0/799630445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0990105360 0x7f0990107750 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0980009900 tx=0x7f098002a410 comp rx=0 tx=0).stop 2024-04-25T12:06:57.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.882+0000 7f09975a8640 1 -- 172.21.15.19:0/799630445 shutdown_connections 2024-04-25T12:06:57.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.882+0000 7f09975a8640 1 --2- 172.21.15.19:0/799630445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0990107c90 0x7f099010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.882+0000 7f09975a8640 1 --2- 172.21.15.19:0/799630445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0990105360 0x7f0990107750 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.882+0000 7f09975a8640 1 --2- 172.21.15.19:0/799630445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0990102a30 0x7f0990104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.882+0000 7f09975a8640 1 -- 172.21.15.19:0/799630445 >> 172.21.15.19:0/799630445 conn(0x7f09900fc440 msgr2=0x7f09900fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:57.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.882+0000 7f09975a8640 1 -- 172.21.15.19:0/799630445 shutdown_connections 2024-04-25T12:06:57.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.882+0000 7f09975a8640 1 -- 172.21.15.19:0/799630445 wait complete. 2024-04-25T12:06:57.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f09975a8640 1 Processor -- start 2024-04-25T12:06:57.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f09975a8640 1 -- start start 2024-04-25T12:06:57.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f09975a8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0990102a30 0x7f09901982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f09975a8640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0990105360 0x7f0990198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f09975a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0990107c90 0x7f099019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f09975a8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f099010dc10 con 0x7f0990107c90 2024-04-25T12:06:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f09975a8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f099010da90 con 0x7f0990102a30 2024-04-25T12:06:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f09975a8640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f099010dd90 con 0x7f0990105360 2024-04-25T12:06:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f0995b45640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0990107c90 0x7f099019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f0995b45640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0990107c90 0x7f099019f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55240/0 (socket says 172.21.15.19:55240) 2024-04-25T12:06:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f0995b45640 1 -- 172.21.15.19:0/1020333001 learned_addr learned my addr 172.21.15.19:0/1020333001 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f0994b43640 1 --2- 172.21.15.19:0/1020333001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0990105360 0x7f0990198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f0995344640 1 --2- 172.21.15.19:0/1020333001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0990102a30 0x7f09901982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f0995b45640 1 -- 172.21.15.19:0/1020333001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0990105360 msgr2=0x7f0990198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f0995b45640 1 --2- 172.21.15.19:0/1020333001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0990105360 0x7f0990198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f0995b45640 1 -- 172.21.15.19:0/1020333001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0990102a30 msgr2=0x7f09901982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f0995b45640 1 --2- 172.21.15.19:0/1020333001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0990102a30 0x7f09901982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f0995b45640 1 -- 172.21.15.19:0/1020333001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099019fee0 con 0x7f0990107c90 2024-04-25T12:06:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f0995b45640 1 --2- 172.21.15.19:0/1020333001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0990107c90 0x7f099019f870 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f098800cce0 tx=0x7f098800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.883+0000 7f09867fc640 1 -- 172.21.15.19:0/1020333001 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098800f450 con 0x7f0990107c90 2024-04-25T12:06:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.884+0000 7f09975a8640 1 -- 172.21.15.19:0/1020333001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09901a01d0 con 0x7f0990107c90 2024-04-25T12:06:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.884+0000 7f09867fc640 1 -- 172.21.15.19:0/1020333001 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f098800fac0 con 0x7f0990107c90 2024-04-25T12:06:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.884+0000 7f09867fc640 1 -- 172.21.15.19:0/1020333001 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988018be0 con 0x7f0990107c90 2024-04-25T12:06:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.884+0000 7f09975a8640 1 -- 172.21.15.19:0/1020333001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09901a06f0 con 0x7f0990107c90 2024-04-25T12:06:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.885+0000 7f09975a8640 1 -- 172.21.15.19:0/1020333001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09900fdf30 con 0x7f0990107c90 2024-04-25T12:06:57.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.886+0000 7f09867fc640 1 -- 172.21.15.19:0/1020333001 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f098800fc70 con 0x7f0990107c90 2024-04-25T12:06:57.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.886+0000 7f09867fc640 1 --2- 172.21.15.19:0/1020333001 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f097c0a3bb0 0x7f097c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:57.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.886+0000 7f09867fc640 1 -- 172.21.15.19:0/1020333001 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f09880d0c50 con 0x7f0990107c90 2024-04-25T12:06:57.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.886+0000 7f0995344640 1 --2- 172.21.15.19:0/1020333001 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f097c0a3bb0 0x7f097c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:57.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.887+0000 7f0995344640 1 --2- 172.21.15.19:0/1020333001 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f097c0a3bb0 0x7f097c0a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f098c005fd0 tx=0x7f098c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:57.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:57.890+0000 7f09867fc640 1 -- 172.21.15.19:0/1020333001 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0988014030 con 0x7f0990107c90 2024-04-25T12:06:58.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:57 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/853267656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:06:58.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:57 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/4281888646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:06:58.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.077+0000 7f09975a8640 1 -- 172.21.15.19:0/1020333001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f099002d5c0 con 0x7f0990107c90 2024-04-25T12:06:58.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.078+0000 7f09867fc640 1 -- 172.21.15.19:0/1020333001 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f098809d6a0 con 0x7f0990107c90 2024-04-25T12:06:58.076 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T12:06:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.079+0000 7f09975a8640 1 -- 172.21.15.19:0/1020333001 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f097c0a3bb0 msgr2=0x7f097c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.079+0000 7f09975a8640 1 --2- 172.21.15.19:0/1020333001 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f097c0a3bb0 0x7f097c0a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f098c005fd0 tx=0x7f098c005850 comp rx=0 tx=0).stop 2024-04-25T12:06:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.079+0000 7f09975a8640 1 -- 172.21.15.19:0/1020333001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0990107c90 msgr2=0x7f099019f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.079+0000 7f09975a8640 1 --2- 172.21.15.19:0/1020333001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0990107c90 0x7f099019f870 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f098800cce0 tx=0x7f098800a620 comp rx=0 tx=0).stop 2024-04-25T12:06:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.080+0000 7f09975a8640 1 -- 172.21.15.19:0/1020333001 shutdown_connections 2024-04-25T12:06:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.080+0000 7f09975a8640 1 --2- 172.21.15.19:0/1020333001 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f097c0a3bb0 0x7f097c0a6070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.080+0000 7f09975a8640 1 --2- 172.21.15.19:0/1020333001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0990107c90 0x7f099019f870 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.080+0000 7f09975a8640 1 --2- 172.21.15.19:0/1020333001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0990105360 0x7f0990198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.080+0000 7f09975a8640 1 --2- 172.21.15.19:0/1020333001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0990102a30 0x7f09901982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.080+0000 7f09975a8640 1 -- 172.21.15.19:0/1020333001 >> 172.21.15.19:0/1020333001 conn(0x7f09900fc440 msgr2=0x7f0990103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:58.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.080+0000 7f09975a8640 1 -- 172.21.15.19:0/1020333001 shutdown_connections 2024-04-25T12:06:58.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.080+0000 7f09975a8640 1 -- 172.21.15.19:0/1020333001 wait complete. 2024-04-25T12:06:58.089 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:06:58.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:57 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/853267656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:06:58.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:57 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/4281888646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:06:58.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:57 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/853267656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:06:58.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:57 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/4281888646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:06:58.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.192+0000 7f454c079640 1 -- 172.21.15.19:0/1776983686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4544100e60 msgr2=0x7f4544101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.192+0000 7f454c079640 1 --2- 172.21.15.19:0/1776983686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4544100e60 0x7f4544101280 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4540009900 tx=0x7f454002a410 comp rx=0 tx=0).stop 2024-04-25T12:06:58.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.192+0000 7f454c079640 1 -- 172.21.15.19:0/1776983686 shutdown_connections 2024-04-25T12:06:58.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.192+0000 7f454c079640 1 --2- 172.21.15.19:0/1776983686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4544102a00 0x7f4544109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.192+0000 7f454c079640 1 --2- 172.21.15.19:0/1776983686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4544102040 0x7f45441024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.192+0000 7f454c079640 1 --2- 172.21.15.19:0/1776983686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4544100e60 0x7f4544101280 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.192+0000 7f454c079640 1 -- 172.21.15.19:0/1776983686 >> 172.21.15.19:0/1776983686 conn(0x7f45440fc440 msgr2=0x7f45440fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.193+0000 7f454c079640 1 -- 172.21.15.19:0/1776983686 shutdown_connections 2024-04-25T12:06:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.193+0000 7f454c079640 1 -- 172.21.15.19:0/1776983686 wait complete. 2024-04-25T12:06:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.193+0000 7f454c079640 1 Processor -- start 2024-04-25T12:06:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.193+0000 7f454c079640 1 -- start start 2024-04-25T12:06:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.193+0000 7f454c079640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4544100e60 0x7f45441982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.193+0000 7f454c079640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4544102040 0x7f45441987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.193+0000 7f454c079640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4544102a00 0x7f454419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.193+0000 7f454c079640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f454410dc40 con 0x7f4544102040 2024-04-25T12:06:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.193+0000 7f454c079640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f454410dac0 con 0x7f4544102a00 2024-04-25T12:06:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.193+0000 7f454c079640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f454410ddc0 con 0x7f4544100e60 2024-04-25T12:06:58.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.193+0000 7f4549614640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4544102040 0x7f45441987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.193+0000 7f4549614640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4544102040 0x7f45441987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55268/0 (socket says 172.21.15.19:55268) 2024-04-25T12:06:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.193+0000 7f4549614640 1 -- 172.21.15.19:0/1331322476 learned_addr learned my addr 172.21.15.19:0/1331322476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.194+0000 7f4549e15640 1 --2- 172.21.15.19:0/1331322476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4544100e60 0x7f45441982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.194+0000 7f4549614640 1 -- 172.21.15.19:0/1331322476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4544100e60 msgr2=0x7f45441982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.194+0000 7f4549614640 1 --2- 172.21.15.19:0/1331322476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4544100e60 0x7f45441982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.194+0000 7f4549614640 1 -- 172.21.15.19:0/1331322476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4544102a00 msgr2=0x7f454419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.194+0000 7f4549614640 1 --2- 172.21.15.19:0/1331322476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4544102a00 0x7f454419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.194+0000 7f4549614640 1 -- 172.21.15.19:0/1331322476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454419fed0 con 0x7f4544102040 2024-04-25T12:06:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.194+0000 7f4549e15640 1 --2- 172.21.15.19:0/1331322476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4544100e60 0x7f45441982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.194+0000 7f4549614640 1 --2- 172.21.15.19:0/1331322476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4544102040 0x7f45441987f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f453400d770 tx=0x7f453400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.194+0000 7f453affd640 1 -- 172.21.15.19:0/1331322476 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534017070 con 0x7f4544102040 2024-04-25T12:06:58.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.194+0000 7f454c079640 1 -- 172.21.15.19:0/1331322476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45441a01c0 con 0x7f4544102040 2024-04-25T12:06:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.194+0000 7f453affd640 1 -- 172.21.15.19:0/1331322476 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f45340054f0 con 0x7f4544102040 2024-04-25T12:06:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.194+0000 7f453affd640 1 -- 172.21.15.19:0/1331322476 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534004de0 con 0x7f4544102040 2024-04-25T12:06:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.194+0000 7f454c079640 1 -- 172.21.15.19:0/1331322476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45441a06c0 con 0x7f4544102040 2024-04-25T12:06:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.195+0000 7f454c079640 1 -- 172.21.15.19:0/1331322476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4544108200 con 0x7f4544102040 2024-04-25T12:06:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.196+0000 7f453affd640 1 -- 172.21.15.19:0/1331322476 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4534004f80 con 0x7f4544102040 2024-04-25T12:06:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.197+0000 7f453affd640 1 --2- 172.21.15.19:0/1331322476 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f45300a3bb0 0x7f45300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.197+0000 7f4549e15640 1 --2- 172.21.15.19:0/1331322476 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f45300a3bb0 0x7f45300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.197+0000 7f453affd640 1 -- 172.21.15.19:0/1331322476 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f45340cffd0 con 0x7f4544102040 2024-04-25T12:06:58.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.197+0000 7f4549e15640 1 --2- 172.21.15.19:0/1331322476 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f45300a3bb0 0x7f45300a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4540000c00 tx=0x7f4540031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:58.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.201+0000 7f453affd640 1 -- 172.21.15.19:0/1331322476 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f453409c990 con 0x7f4544102040 2024-04-25T12:06:58.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.427+0000 7f454c079640 1 -- 172.21.15.19:0/1331322476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4544062b50 con 0x7f4544102040 2024-04-25T12:06:58.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.427+0000 7f453affd640 1 -- 172.21.15.19:0/1331322476 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11944 (secure 0 0 0) 0x7f453409c290 con 0x7f4544102040 2024-04-25T12:06:58.426 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:58.426 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T12:05:04.008679+0000","modified":"2024-04-25T12:05:11.186222+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24514,"mds_1":24500,"mds_2":24496,"mds_3":24484,"mds_4":14655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/1694897630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1694897630},{"type":"v1","addr":"172.21.15.19:6839","nonce":1694897630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6841/3247157172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3247157172},{"type":"v1","addr":"172.21.15.19:6841","nonce":3247157172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/3918801450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3918801450},{"type":"v1","addr":"172.21.15.150:6837","nonce":3918801450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2553528303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2553528303},{"type":"v1","addr":"172.21.15.150:6839","nonce":2553528303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/1723439500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1723439500},{"type":"v1","addr":"172.21.15.169:6839","nonce":1723439500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24500,24514,24496,24484,14655]},"id":1}]} 2024-04-25T12:06:58.426 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-25T12:06:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.429+0000 7f454c079640 1 -- 172.21.15.19:0/1331322476 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f45300a3bb0 msgr2=0x7f45300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.429+0000 7f454c079640 1 --2- 172.21.15.19:0/1331322476 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f45300a3bb0 0x7f45300a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4540000c00 tx=0x7f4540031510 comp rx=0 tx=0).stop 2024-04-25T12:06:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.429+0000 7f454c079640 1 -- 172.21.15.19:0/1331322476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4544102040 msgr2=0x7f45441987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.429+0000 7f454c079640 1 --2- 172.21.15.19:0/1331322476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4544102040 0x7f45441987f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f453400d770 tx=0x7f453400dc30 comp rx=0 tx=0).stop 2024-04-25T12:06:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.430+0000 7f454c079640 1 -- 172.21.15.19:0/1331322476 shutdown_connections 2024-04-25T12:06:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.430+0000 7f454c079640 1 --2- 172.21.15.19:0/1331322476 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f45300a3bb0 0x7f45300a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.430+0000 7f454c079640 1 --2- 172.21.15.19:0/1331322476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4544102a00 0x7f454419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.430+0000 7f454c079640 1 --2- 172.21.15.19:0/1331322476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4544102040 0x7f45441987f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.430+0000 7f454c079640 1 --2- 172.21.15.19:0/1331322476 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4544100e60 0x7f45441982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.430+0000 7f454c079640 1 -- 172.21.15.19:0/1331322476 >> 172.21.15.19:0/1331322476 conn(0x7f45440fc440 msgr2=0x7f45441053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:58.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.430+0000 7f454c079640 1 -- 172.21.15.19:0/1331322476 shutdown_connections 2024-04-25T12:06:58.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.430+0000 7f454c079640 1 -- 172.21.15.19:0/1331322476 wait complete. 2024-04-25T12:06:58.442 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:06:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.546+0000 7fd93697d640 1 -- 172.21.15.19:0/2525141037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd930100880 msgr2=0x7fd930100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.546+0000 7fd93697d640 1 --2- 172.21.15.19:0/2525141037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd930100880 0x7fd930100ca0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd920009900 tx=0x7fd92002a480 comp rx=0 tx=0).stop 2024-04-25T12:06:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.546+0000 7fd93697d640 1 -- 172.21.15.19:0/2525141037 shutdown_connections 2024-04-25T12:06:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.546+0000 7fd93697d640 1 --2- 172.21.15.19:0/2525141037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd930101b80 0x7fd930102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.546+0000 7fd93697d640 1 --2- 172.21.15.19:0/2525141037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9301011e0 0x7fd930101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.546+0000 7fd93697d640 1 --2- 172.21.15.19:0/2525141037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd930100880 0x7fd930100ca0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.546+0000 7fd93697d640 1 -- 172.21.15.19:0/2525141037 >> 172.21.15.19:0/2525141037 conn(0x7fd9300fc400 msgr2=0x7fd9300fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.546+0000 7fd93697d640 1 -- 172.21.15.19:0/2525141037 shutdown_connections 2024-04-25T12:06:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.546+0000 7fd93697d640 1 -- 172.21.15.19:0/2525141037 wait complete. 2024-04-25T12:06:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.547+0000 7fd93697d640 1 Processor -- start 2024-04-25T12:06:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.547+0000 7fd93697d640 1 -- start start 2024-04-25T12:06:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.547+0000 7fd93697d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd930100880 0x7fd93019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.547+0000 7fd93697d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9301011e0 0x7fd93019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.547+0000 7fd93697d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd930101b80 0x7fd9301a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.547+0000 7fd93697d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd93010fdf0 con 0x7fd930101b80 2024-04-25T12:06:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.547+0000 7fd93697d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd93010fc70 con 0x7fd9301011e0 2024-04-25T12:06:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.547+0000 7fd93697d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd93010ff70 con 0x7fd930100880 2024-04-25T12:06:58.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.547+0000 7fd934f1a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd930101b80 0x7fd9301a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:58.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.547+0000 7fd934f1a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd930101b80 0x7fd9301a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55278/0 (socket says 172.21.15.19:55278) 2024-04-25T12:06:58.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.547+0000 7fd934f1a640 1 -- 172.21.15.19:0/1204841546 learned_addr learned my addr 172.21.15.19:0/1204841546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:58.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.548+0000 7fd92f7fe640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9301011e0 0x7fd93019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:58.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.548+0000 7fd92ffff640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd930100880 0x7fd93019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:58.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.548+0000 7fd934f1a640 1 -- 172.21.15.19:0/1204841546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd930100880 msgr2=0x7fd93019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.548+0000 7fd934f1a640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd930100880 0x7fd93019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.548+0000 7fd934f1a640 1 -- 172.21.15.19:0/1204841546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9301011e0 msgr2=0x7fd93019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.548+0000 7fd934f1a640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9301011e0 0x7fd93019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.548+0000 7fd934f1a640 1 -- 172.21.15.19:0/1204841546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9301a20f0 con 0x7fd930101b80 2024-04-25T12:06:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.548+0000 7fd92f7fe640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9301011e0 0x7fd93019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:06:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.548+0000 7fd934f1a640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd930101b80 0x7fd9301a1a80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd928009fd0 tx=0x7fd92800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.548+0000 7fd92d7fa640 1 -- 172.21.15.19:0/1204841546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928017070 con 0x7fd930101b80 2024-04-25T12:06:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.548+0000 7fd92d7fa640 1 -- 172.21.15.19:0/1204841546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd928003e40 con 0x7fd930101b80 2024-04-25T12:06:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.548+0000 7fd93697d640 1 -- 172.21.15.19:0/1204841546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9301a23e0 con 0x7fd930101b80 2024-04-25T12:06:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.548+0000 7fd92d7fa640 1 -- 172.21.15.19:0/1204841546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928004820 con 0x7fd930101b80 2024-04-25T12:06:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.548+0000 7fd93697d640 1 -- 172.21.15.19:0/1204841546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9301a2930 con 0x7fd930101b80 2024-04-25T12:06:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.549+0000 7fd93697d640 1 -- 172.21.15.19:0/1204841546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd93010b7a0 con 0x7fd930101b80 2024-04-25T12:06:58.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.550+0000 7fd92d7fa640 1 -- 172.21.15.19:0/1204841546 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd928003970 con 0x7fd930101b80 2024-04-25T12:06:58.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.551+0000 7fd92d7fa640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd9180a3c30 0x7fd9180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:58.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.551+0000 7fd92ffff640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd9180a3c30 0x7fd9180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:58.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.551+0000 7fd92d7fa640 1 -- 172.21.15.19:0/1204841546 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd9280d02b0 con 0x7fd930101b80 2024-04-25T12:06:58.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.552+0000 7fd92ffff640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd9180a3c30 0x7fd9180a60f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd920000c00 tx=0x7fd920031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:58.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.555+0000 7fd92d7fa640 1 -- 172.21.15.19:0/1204841546 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd928003c60 con 0x7fd930101b80 2024-04-25T12:06:58.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.744+0000 7fd93697d640 1 -- 172.21.15.19:0/1204841546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd930062b50 con 0x7fd930101b80 2024-04-25T12:06:58.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.745+0000 7fd92d7fa640 1 -- 172.21.15.19:0/1204841546 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22731 (secure 0 0 0) 0x7fd92809cc90 con 0x7fd930101b80 2024-04-25T12:06:58.744 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:06:58.744 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":87,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:05:55.975412+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:05:01.658354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:05:02.969078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:06:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.747+0000 7fd93697d640 1 -- 172.21.15.19:0/1204841546 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd9180a3c30 msgr2=0x7fd9180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.747+0000 7fd93697d640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd9180a3c30 0x7fd9180a60f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd920000c00 tx=0x7fd920031510 comp rx=0 tx=0).stop 2024-04-25T12:06:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.747+0000 7fd93697d640 1 -- 172.21.15.19:0/1204841546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd930101b80 msgr2=0x7fd9301a1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.747+0000 7fd93697d640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd930101b80 0x7fd9301a1a80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd928009fd0 tx=0x7fd92800b260 comp rx=0 tx=0).stop 2024-04-25T12:06:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.748+0000 7fd93697d640 1 -- 172.21.15.19:0/1204841546 shutdown_connections 2024-04-25T12:06:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.748+0000 7fd93697d640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fd9180a3c30 0x7fd9180a60f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.748+0000 7fd93697d640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd930101b80 0x7fd9301a1a80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.748+0000 7fd93697d640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9301011e0 0x7fd93019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.748+0000 7fd93697d640 1 --2- 172.21.15.19:0/1204841546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd930100880 0x7fd93019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.748+0000 7fd93697d640 1 -- 172.21.15.19:0/1204841546 >> 172.21.15.19:0/1204841546 conn(0x7fd9300fc400 msgr2=0x7fd9300fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.748+0000 7fd93697d640 1 -- 172.21.15.19:0/1204841546 shutdown_connections 2024-04-25T12:06:58.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.748+0000 7fd93697d640 1 -- 172.21.15.19:0/1204841546 wait complete. 2024-04-25T12:06:58.757 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:00:20.898809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:05:01.658354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:05:02.969078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T12:06:58.758 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:06:58.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.864+0000 7f4629522640 1 -- 172.21.15.19:0/3429404752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4624102040 msgr2=0x7f46241024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.864+0000 7f4629522640 1 --2- 172.21.15.19:0/3429404752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4624102040 0x7f46241024c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4618009900 tx=0x7f461802a410 comp rx=0 tx=0).stop 2024-04-25T12:06:58.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.864+0000 7f4629522640 1 -- 172.21.15.19:0/3429404752 shutdown_connections 2024-04-25T12:06:58.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.864+0000 7f4629522640 1 --2- 172.21.15.19:0/3429404752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4624102a00 0x7f4624109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.864+0000 7f4629522640 1 --2- 172.21.15.19:0/3429404752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4624102040 0x7f46241024c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.864+0000 7f4629522640 1 --2- 172.21.15.19:0/3429404752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4624100e60 0x7f4624101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.864+0000 7f4629522640 1 -- 172.21.15.19:0/3429404752 >> 172.21.15.19:0/3429404752 conn(0x7f46240fc440 msgr2=0x7f46240fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:58.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.864+0000 7f4629522640 1 -- 172.21.15.19:0/3429404752 shutdown_connections 2024-04-25T12:06:58.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.864+0000 7f4629522640 1 -- 172.21.15.19:0/3429404752 wait complete. 2024-04-25T12:06:58.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f4629522640 1 Processor -- start 2024-04-25T12:06:58.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f4629522640 1 -- start start 2024-04-25T12:06:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f4629522640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4624100e60 0x7f462406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f4629522640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4624102040 0x7f462419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f4629522640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4624102a00 0x7f462419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f4629522640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f462410cb30 con 0x7f4624102040 2024-04-25T12:06:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f4629522640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f462410c9b0 con 0x7f4624100e60 2024-04-25T12:06:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f4629522640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f462410ccb0 con 0x7f4624102a00 2024-04-25T12:06:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f46227fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4624102040 0x7f462419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f46227fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4624102040 0x7f462419cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55296/0 (socket says 172.21.15.19:55296) 2024-04-25T12:06:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f46227fc640 1 -- 172.21.15.19:0/921415252 learned_addr learned my addr 172.21.15.19:0/921415252 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f46237fe640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4624102a00 0x7f462419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f4622ffd640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4624100e60 0x7f462406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f46227fc640 1 -- 172.21.15.19:0/921415252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4624102a00 msgr2=0x7f462419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f46227fc640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4624102a00 0x7f462419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f46227fc640 1 -- 172.21.15.19:0/921415252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4624100e60 msgr2=0x7f462406b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f46227fc640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4624100e60 0x7f462406b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f46227fc640 1 -- 172.21.15.19:0/921415252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462419fec0 con 0x7f4624102040 2024-04-25T12:06:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f4622ffd640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4624100e60 0x7f462406b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:06:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.865+0000 7f46227fc640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4624102040 0x7f462419cf90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4618005940 tx=0x7f46180043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.866+0000 7f460bfff640 1 -- 172.21.15.19:0/921415252 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461803f030 con 0x7f4624102040 2024-04-25T12:06:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.866+0000 7f4629522640 1 -- 172.21.15.19:0/921415252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46241a00f0 con 0x7f4624102040 2024-04-25T12:06:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.866+0000 7f460bfff640 1 -- 172.21.15.19:0/921415252 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4618003cd0 con 0x7f4624102040 2024-04-25T12:06:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.866+0000 7f460bfff640 1 -- 172.21.15.19:0/921415252 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618035070 con 0x7f4624102040 2024-04-25T12:06:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.866+0000 7f4629522640 1 -- 172.21.15.19:0/921415252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46241a0630 con 0x7f4624102040 2024-04-25T12:06:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.867+0000 7f4629522640 1 -- 172.21.15.19:0/921415252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4624108200 con 0x7f4624102040 2024-04-25T12:06:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.868+0000 7f460bfff640 1 -- 172.21.15.19:0/921415252 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4618042030 con 0x7f4624102040 2024-04-25T12:06:58.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.868+0000 7f460bfff640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f460c0a3bb0 0x7f460c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:58.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.868+0000 7f4622ffd640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f460c0a3bb0 0x7f460c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:58.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.868+0000 7f460bfff640 1 -- 172.21.15.19:0/921415252 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f46180f0ae0 con 0x7f4624102040 2024-04-25T12:06:58.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.869+0000 7f4622ffd640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f460c0a3bb0 0x7f460c0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4624101bb0 tx=0x7f4614005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:58.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:58.873+0000 7f460bfff640 1 -- 172.21.15.19:0/921415252 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46180bd4a0 con 0x7f4624102040 2024-04-25T12:06:59.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:58 smithi019 ceph-mon[29213]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T12:06:59.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:58 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1626758495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:06:59.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:58 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1020333001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:06:59.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:58 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1331322476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:06:59.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.059+0000 7f4629522640 1 -- 172.21.15.19:0/921415252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4624062b50 con 0x7f4624102040 2024-04-25T12:06:59.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.059+0000 7f460bfff640 1 -- 172.21.15.19:0/921415252 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f46180bcda0 con 0x7f4624102040 2024-04-25T12:06:59.059 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T12:06:59.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.061+0000 7f4629522640 1 -- 172.21.15.19:0/921415252 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f460c0a3bb0 msgr2=0x7f460c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.061+0000 7f4629522640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f460c0a3bb0 0x7f460c0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4624101bb0 tx=0x7f4614005db0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.061+0000 7f4629522640 1 -- 172.21.15.19:0/921415252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4624102040 msgr2=0x7f462419cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.061+0000 7f4629522640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4624102040 0x7f462419cf90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4618005940 tx=0x7f46180043b0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.061+0000 7f4629522640 1 -- 172.21.15.19:0/921415252 shutdown_connections 2024-04-25T12:06:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.061+0000 7f4629522640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f460c0a3bb0 0x7f460c0a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.061+0000 7f4629522640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4624102a00 0x7f462419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.061+0000 7f4629522640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4624102040 0x7f462419cf90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.061+0000 7f4629522640 1 --2- 172.21.15.19:0/921415252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4624100e60 0x7f462406b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.061+0000 7f4629522640 1 -- 172.21.15.19:0/921415252 >> 172.21.15.19:0/921415252 conn(0x7f46240fc440 msgr2=0x7f46241053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.061+0000 7f4629522640 1 -- 172.21.15.19:0/921415252 shutdown_connections 2024-04-25T12:06:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.061+0000 7f4629522640 1 -- 172.21.15.19:0/921415252 wait complete. 2024-04-25T12:06:59.071 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T12:06:59.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:58 smithi169 ceph-mon[32710]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T12:06:59.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:58 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1626758495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:06:59.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:58 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1020333001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:06:59.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:58 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1331322476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:06:59.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:58 smithi150 ceph-mon[32250]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T12:06:59.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:58 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1626758495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:06:59.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:58 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1020333001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:06:59.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:58 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1331322476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:06:59.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.178+0000 7f188b507640 1 -- 172.21.15.19:0/2102874488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1884101fe0 msgr2=0x7f1884102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.178+0000 7f188b507640 1 --2- 172.21.15.19:0/2102874488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1884101fe0 0x7f1884102460 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1874009900 tx=0x7f187402a410 comp rx=0 tx=0).stop 2024-04-25T12:06:59.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.178+0000 7f188b507640 1 -- 172.21.15.19:0/2102874488 shutdown_connections 2024-04-25T12:06:59.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.178+0000 7f188b507640 1 --2- 172.21.15.19:0/2102874488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18841029a0 0x7f18841092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.178+0000 7f188b507640 1 --2- 172.21.15.19:0/2102874488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1884101fe0 0x7f1884102460 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.178+0000 7f188b507640 1 --2- 172.21.15.19:0/2102874488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1884100e90 0x7f18841012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.178+0000 7f188b507640 1 -- 172.21.15.19:0/2102874488 >> 172.21.15.19:0/2102874488 conn(0x7f18840fc400 msgr2=0x7f18840fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.178+0000 7f188b507640 1 -- 172.21.15.19:0/2102874488 shutdown_connections 2024-04-25T12:06:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.178+0000 7f188b507640 1 -- 172.21.15.19:0/2102874488 wait complete. 2024-04-25T12:06:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.178+0000 7f188b507640 1 Processor -- start 2024-04-25T12:06:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.178+0000 7f188b507640 1 -- start start 2024-04-25T12:06:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.178+0000 7f188b507640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1884100e90 0x7f188406b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f188b507640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1884101fe0 0x7f188419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f188b507640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18841029a0 0x7f188419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f188b507640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f188410cb30 con 0x7f18841029a0 2024-04-25T12:06:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f188b507640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f188410c9b0 con 0x7f1884100e90 2024-04-25T12:06:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f188b507640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f188410ccb0 con 0x7f1884101fe0 2024-04-25T12:06:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f1889aa4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18841029a0 0x7f188419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:59.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f1889aa4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18841029a0 0x7f188419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55320/0 (socket says 172.21.15.19:55320) 2024-04-25T12:06:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f1889aa4640 1 -- 172.21.15.19:0/1041319795 learned_addr learned my addr 172.21.15.19:0/1041319795 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f18892a3640 1 --2- 172.21.15.19:0/1041319795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1884100e90 0x7f188406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f1889aa4640 1 -- 172.21.15.19:0/1041319795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1884101fe0 msgr2=0x7f188419d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f1889aa4640 1 --2- 172.21.15.19:0/1041319795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1884101fe0 0x7f188419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f1889aa4640 1 -- 172.21.15.19:0/1041319795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1884100e90 msgr2=0x7f188406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f1889aa4640 1 --2- 172.21.15.19:0/1041319795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1884100e90 0x7f188406b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f1889aa4640 1 -- 172.21.15.19:0/1041319795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188419ff30 con 0x7f18841029a0 2024-04-25T12:06:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f18892a3640 1 --2- 172.21.15.19:0/1041319795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1884100e90 0x7f188406b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f1889aa4640 1 --2- 172.21.15.19:0/1041319795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18841029a0 0x7f188419f620 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f187c00eaa0 tx=0x7f187c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f187a7fc640 1 -- 172.21.15.19:0/1041319795 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c00cdb0 con 0x7f18841029a0 2024-04-25T12:06:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f187a7fc640 1 -- 172.21.15.19:0/1041319795 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f187c005020 con 0x7f18841029a0 2024-04-25T12:06:59.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f188b507640 1 -- 172.21.15.19:0/1041319795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18841a01c0 con 0x7f18841029a0 2024-04-25T12:06:59.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f187a7fc640 1 -- 172.21.15.19:0/1041319795 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c0116d0 con 0x7f18841029a0 2024-04-25T12:06:59.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.179+0000 7f188b507640 1 -- 172.21.15.19:0/1041319795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18841a0710 con 0x7f18841029a0 2024-04-25T12:06:59.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.180+0000 7f188b507640 1 -- 172.21.15.19:0/1041319795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18841081a0 con 0x7f18841029a0 2024-04-25T12:06:59.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.181+0000 7f187a7fc640 1 -- 172.21.15.19:0/1041319795 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f187c0051d0 con 0x7f18841029a0 2024-04-25T12:06:59.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.182+0000 7f187a7fc640 1 --2- 172.21.15.19:0/1041319795 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f18700a3c30 0x7f18700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:59.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.182+0000 7f187a7fc640 1 -- 172.21.15.19:0/1041319795 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f187c0d06e0 con 0x7f18841029a0 2024-04-25T12:06:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.182+0000 7f18892a3640 1 --2- 172.21.15.19:0/1041319795 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f18700a3c30 0x7f18700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:59.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.182+0000 7f18892a3640 1 --2- 172.21.15.19:0/1041319795 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f18700a3c30 0x7f18700a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f18800058c0 tx=0x7f1880005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:59.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.186+0000 7f187a7fc640 1 -- 172.21.15.19:0/1041319795 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f187c016030 con 0x7f18841029a0 2024-04-25T12:06:59.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.373+0000 7f188b507640 1 -- 172.21.15.19:0/1041319795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1884062b50 con 0x7f18841029a0 2024-04-25T12:06:59.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.374+0000 7f187a7fc640 1 -- 172.21.15.19:0/1041319795 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f187c09d0a0 con 0x7f18841029a0 2024-04-25T12:06:59.372 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T12:06:59.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.376+0000 7f188b507640 1 -- 172.21.15.19:0/1041319795 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f18700a3c30 msgr2=0x7f18700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.376+0000 7f188b507640 1 --2- 172.21.15.19:0/1041319795 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f18700a3c30 0x7f18700a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f18800058c0 tx=0x7f1880005850 comp rx=0 tx=0).stop 2024-04-25T12:06:59.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.376+0000 7f188b507640 1 -- 172.21.15.19:0/1041319795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18841029a0 msgr2=0x7f188419f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.376+0000 7f188b507640 1 --2- 172.21.15.19:0/1041319795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18841029a0 0x7f188419f620 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f187c00eaa0 tx=0x7f187c00ef60 comp rx=0 tx=0).stop 2024-04-25T12:06:59.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.376+0000 7f188b507640 1 -- 172.21.15.19:0/1041319795 shutdown_connections 2024-04-25T12:06:59.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.376+0000 7f188b507640 1 --2- 172.21.15.19:0/1041319795 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f18700a3c30 0x7f18700a60f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.376+0000 7f188b507640 1 --2- 172.21.15.19:0/1041319795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f18841029a0 0x7f188419f620 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.376+0000 7f188b507640 1 --2- 172.21.15.19:0/1041319795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1884101fe0 0x7f188419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.376+0000 7f188b507640 1 --2- 172.21.15.19:0/1041319795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1884100e90 0x7f188406b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.376+0000 7f188b507640 1 -- 172.21.15.19:0/1041319795 >> 172.21.15.19:0/1041319795 conn(0x7f18840fc400 msgr2=0x7f1884105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:59.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.376+0000 7f188b507640 1 -- 172.21.15.19:0/1041319795 shutdown_connections 2024-04-25T12:06:59.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.376+0000 7f188b507640 1 -- 172.21.15.19:0/1041319795 wait complete. 2024-04-25T12:06:59.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T12:06:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.492+0000 7f06872cf640 1 -- 172.21.15.19:0/719939159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0680100e60 msgr2=0x7f0680101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.492+0000 7f06872cf640 1 --2- 172.21.15.19:0/719939159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0680100e60 0x7f0680101280 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f067c009900 tx=0x7f067c02a410 comp rx=0 tx=0).stop 2024-04-25T12:06:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.492+0000 7f06872cf640 1 -- 172.21.15.19:0/719939159 shutdown_connections 2024-04-25T12:06:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.492+0000 7f06872cf640 1 --2- 172.21.15.19:0/719939159 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680102a00 0x7f0680109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.492+0000 7f06872cf640 1 --2- 172.21.15.19:0/719939159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0680102040 0x7f06801024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.492+0000 7f06872cf640 1 --2- 172.21.15.19:0/719939159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0680100e60 0x7f0680101280 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.492+0000 7f06872cf640 1 -- 172.21.15.19:0/719939159 >> 172.21.15.19:0/719939159 conn(0x7f06800fc440 msgr2=0x7f06800fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.492+0000 7f06872cf640 1 -- 172.21.15.19:0/719939159 shutdown_connections 2024-04-25T12:06:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.492+0000 7f06872cf640 1 -- 172.21.15.19:0/719939159 wait complete. 2024-04-25T12:06:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f06872cf640 1 Processor -- start 2024-04-25T12:06:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f06872cf640 1 -- start start 2024-04-25T12:06:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f06872cf640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680100e60 0x7f0680104db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f06872cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0680102040 0x7f06801034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f06872cf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0680102a00 0x7f06801039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f06872cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f068010cb60 con 0x7f0680102040 2024-04-25T12:06:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f06872cf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f068010c9e0 con 0x7f0680102a00 2024-04-25T12:06:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f06872cf640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f068010cce0 con 0x7f0680100e60 2024-04-25T12:06:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f068486a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0680102040 0x7f06801034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:59.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f068486a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0680102040 0x7f06801034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55330/0 (socket says 172.21.15.19:55330) 2024-04-25T12:06:59.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f068486a640 1 -- 172.21.15.19:0/1674470255 learned_addr learned my addr 172.21.15.19:0/1674470255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:59.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f068586c640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0680102a00 0x7f06801039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:59.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f068506b640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680100e60 0x7f0680104db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:59.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f068486a640 1 -- 172.21.15.19:0/1674470255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680100e60 msgr2=0x7f0680104db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f068486a640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680100e60 0x7f0680104db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f068486a640 1 -- 172.21.15.19:0/1674470255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0680102a00 msgr2=0x7f06801039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f068486a640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0680102a00 0x7f06801039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f068486a640 1 -- 172.21.15.19:0/1674470255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06801041c0 con 0x7f0680102040 2024-04-25T12:06:59.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.493+0000 7f068506b640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680100e60 0x7f0680104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:06:59.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.494+0000 7f068486a640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0680102040 0x7f06801034b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f067000d950 tx=0x7f067000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:59.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.494+0000 7f06767fc640 1 -- 172.21.15.19:0/1674470255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0670017070 con 0x7f0680102040 2024-04-25T12:06:59.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.494+0000 7f06767fc640 1 -- 172.21.15.19:0/1674470255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0670009c30 con 0x7f0680102040 2024-04-25T12:06:59.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.494+0000 7f06872cf640 1 -- 172.21.15.19:0/1674470255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06801a2410 con 0x7f0680102040 2024-04-25T12:06:59.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.494+0000 7f06767fc640 1 -- 172.21.15.19:0/1674470255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067000f6b0 con 0x7f0680102040 2024-04-25T12:06:59.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.494+0000 7f06872cf640 1 -- 172.21.15.19:0/1674470255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06801a2960 con 0x7f0680102040 2024-04-25T12:06:59.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.495+0000 7f06872cf640 1 -- 172.21.15.19:0/1674470255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06801a2bc0 con 0x7f0680102040 2024-04-25T12:06:59.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.496+0000 7f06767fc640 1 -- 172.21.15.19:0/1674470255 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f06700109c0 con 0x7f0680102040 2024-04-25T12:06:59.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.496+0000 7f06767fc640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f066c0a3bb0 0x7f066c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:59.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.496+0000 7f068506b640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f066c0a3bb0 0x7f066c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:59.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.497+0000 7f06767fc640 1 -- 172.21.15.19:0/1674470255 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f06700d0050 con 0x7f0680102040 2024-04-25T12:06:59.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.497+0000 7f068506b640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f066c0a3bb0 0x7f066c0a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f067c006010 tx=0x7f067c031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:59.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.501+0000 7f06767fc640 1 -- 172.21.15.19:0/1674470255 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0670009de0 con 0x7f0680102040 2024-04-25T12:06:59.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.690+0000 7f06872cf640 1 -- 172.21.15.19:0/1674470255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f06801a2d90 con 0x7f0680102040 2024-04-25T12:06:59.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.690+0000 7f06767fc640 1 -- 172.21.15.19:0/1674470255 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f067009ca10 con 0x7f0680102040 2024-04-25T12:06:59.689 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T12:06:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.692+0000 7f06872cf640 1 -- 172.21.15.19:0/1674470255 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f066c0a3bb0 msgr2=0x7f066c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.692+0000 7f06872cf640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f066c0a3bb0 0x7f066c0a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f067c006010 tx=0x7f067c031510 comp rx=0 tx=0).stop 2024-04-25T12:06:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.692+0000 7f06872cf640 1 -- 172.21.15.19:0/1674470255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0680102040 msgr2=0x7f06801034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.692+0000 7f06872cf640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0680102040 0x7f06801034b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f067000d950 tx=0x7f067000de10 comp rx=0 tx=0).stop 2024-04-25T12:06:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.692+0000 7f06872cf640 1 -- 172.21.15.19:0/1674470255 shutdown_connections 2024-04-25T12:06:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.692+0000 7f06872cf640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f066c0a3bb0 0x7f066c0a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.692+0000 7f06872cf640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0680102a00 0x7f06801039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.692+0000 7f06872cf640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0680102040 0x7f06801034b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.693+0000 7f06872cf640 1 --2- 172.21.15.19:0/1674470255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0680100e60 0x7f0680104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.693+0000 7f06872cf640 1 -- 172.21.15.19:0/1674470255 >> 172.21.15.19:0/1674470255 conn(0x7f06800fc440 msgr2=0x7f06800fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:59.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.693+0000 7f06872cf640 1 -- 172.21.15.19:0/1674470255 shutdown_connections 2024-04-25T12:06:59.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.693+0000 7f06872cf640 1 -- 172.21.15.19:0/1674470255 wait complete. 2024-04-25T12:06:59.702 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:06:59.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.806+0000 7fb953aa7640 1 -- 172.21.15.19:0/2460690248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c100e90 msgr2=0x7fb94c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.806+0000 7fb953aa7640 1 --2- 172.21.15.19:0/2460690248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c100e90 0x7fb94c1012b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb948009900 tx=0x7fb94802a410 comp rx=0 tx=0).stop 2024-04-25T12:06:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.806+0000 7fb953aa7640 1 -- 172.21.15.19:0/2460690248 shutdown_connections 2024-04-25T12:06:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.806+0000 7fb953aa7640 1 --2- 172.21.15.19:0/2460690248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c1029a0 0x7fb94c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.806+0000 7fb953aa7640 1 --2- 172.21.15.19:0/2460690248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb94c101fe0 0x7fb94c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.806+0000 7fb953aa7640 1 --2- 172.21.15.19:0/2460690248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c100e90 0x7fb94c1012b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.806+0000 7fb953aa7640 1 -- 172.21.15.19:0/2460690248 >> 172.21.15.19:0/2460690248 conn(0x7fb94c0fc400 msgr2=0x7fb94c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:06:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.806+0000 7fb953aa7640 1 -- 172.21.15.19:0/2460690248 shutdown_connections 2024-04-25T12:06:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.806+0000 7fb953aa7640 1 -- 172.21.15.19:0/2460690248 wait complete. 2024-04-25T12:06:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.807+0000 7fb953aa7640 1 Processor -- start 2024-04-25T12:06:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.807+0000 7fb953aa7640 1 -- start start 2024-04-25T12:06:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.807+0000 7fb953aa7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c100e90 0x7fb94c104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.807+0000 7fb953aa7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c101fe0 0x7fb94c103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:59.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.807+0000 7fb953aa7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb94c1029a0 0x7fb94c103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.807+0000 7fb953aa7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c10cba0 con 0x7fb94c101fe0 2024-04-25T12:06:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.807+0000 7fb953aa7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c10ca20 con 0x7fb94c100e90 2024-04-25T12:06:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.807+0000 7fb953aa7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c10cd20 con 0x7fb94c1029a0 2024-04-25T12:06:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.807+0000 7fb951042640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c101fe0 0x7fb94c103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.807+0000 7fb951042640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c101fe0 0x7fb94c103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55352/0 (socket says 172.21.15.19:55352) 2024-04-25T12:06:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.807+0000 7fb951042640 1 -- 172.21.15.19:0/324282005 learned_addr learned my addr 172.21.15.19:0/324282005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:06:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.807+0000 7fb951843640 1 --2- 172.21.15.19:0/324282005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c100e90 0x7fb94c104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.808+0000 7fb952044640 1 --2- 172.21.15.19:0/324282005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb94c1029a0 0x7fb94c103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.808+0000 7fb951042640 1 -- 172.21.15.19:0/324282005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb94c1029a0 msgr2=0x7fb94c103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.808+0000 7fb951042640 1 --2- 172.21.15.19:0/324282005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb94c1029a0 0x7fb94c103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.808+0000 7fb951042640 1 -- 172.21.15.19:0/324282005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c100e90 msgr2=0x7fb94c104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:06:59.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.808+0000 7fb951042640 1 --2- 172.21.15.19:0/324282005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c100e90 0x7fb94c104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:06:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.808+0000 7fb951042640 1 -- 172.21.15.19:0/324282005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94c104220 con 0x7fb94c101fe0 2024-04-25T12:06:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.808+0000 7fb951042640 1 --2- 172.21.15.19:0/324282005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c101fe0 0x7fb94c103450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00b450 tx=0x7fb93c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.808+0000 7fb942ffd640 1 -- 172.21.15.19:0/324282005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c015020 con 0x7fb94c101fe0 2024-04-25T12:06:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.808+0000 7fb953aa7640 1 -- 172.21.15.19:0/324282005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94c1a24a0 con 0x7fb94c101fe0 2024-04-25T12:06:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.808+0000 7fb942ffd640 1 -- 172.21.15.19:0/324282005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb93c011d60 con 0x7fb94c101fe0 2024-04-25T12:06:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.808+0000 7fb942ffd640 1 -- 172.21.15.19:0/324282005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00f340 con 0x7fb94c101fe0 2024-04-25T12:06:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.808+0000 7fb953aa7640 1 -- 172.21.15.19:0/324282005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94c1a2a70 con 0x7fb94c101fe0 2024-04-25T12:06:59.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.809+0000 7fb953aa7640 1 -- 172.21.15.19:0/324282005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94c1a2d30 con 0x7fb94c101fe0 2024-04-25T12:06:59.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.810+0000 7fb942ffd640 1 -- 172.21.15.19:0/324282005 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb93c018020 con 0x7fb94c101fe0 2024-04-25T12:06:59.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.811+0000 7fb942ffd640 1 --2- 172.21.15.19:0/324282005 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb9380a3bb0 0x7fb9380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:06:59.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.811+0000 7fb951843640 1 --2- 172.21.15.19:0/324282005 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb9380a3bb0 0x7fb9380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:06:59.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.811+0000 7fb942ffd640 1 -- 172.21.15.19:0/324282005 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fb93c0cff80 con 0x7fb94c101fe0 2024-04-25T12:06:59.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.811+0000 7fb951843640 1 --2- 172.21.15.19:0/324282005 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb9380a3bb0 0x7fb9380a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb948000c00 tx=0x7fb948031510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:06:59.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:06:59.815+0000 7fb942ffd640 1 -- 172.21.15.19:0/324282005 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb93c01f020 con 0x7fb94c101fe0 2024-04-25T12:07:00.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.041+0000 7fb953aa7640 1 -- 172.21.15.19:0/324282005 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb94c1a30c0 con 0x7fb94c101fe0 2024-04-25T12:07:00.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.042+0000 7fb942ffd640 1 -- 172.21.15.19:0/324282005 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11944 (secure 0 0 0) 0x7fb93c00d040 con 0x7fb94c101fe0 2024-04-25T12:07:00.040 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:07:00.040 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T12:05:04.008679+0000","modified":"2024-04-25T12:05:11.186222+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24514,"mds_1":24500,"mds_2":24496,"mds_3":24484,"mds_4":14655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/1694897630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1694897630},{"type":"v1","addr":"172.21.15.19:6839","nonce":1694897630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6841/3247157172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3247157172},{"type":"v1","addr":"172.21.15.19:6841","nonce":3247157172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/3918801450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3918801450},{"type":"v1","addr":"172.21.15.150:6837","nonce":3918801450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2553528303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2553528303},{"type":"v1","addr":"172.21.15.150:6839","nonce":2553528303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/1723439500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1723439500},{"type":"v1","addr":"172.21.15.169:6839","nonce":1723439500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24500,24514,24496,24484,14655]},"id":1}]} 2024-04-25T12:07:00.040 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-04-25T12:07:00.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.044+0000 7fb953aa7640 1 -- 172.21.15.19:0/324282005 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb9380a3bb0 msgr2=0x7fb9380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.044+0000 7fb953aa7640 1 --2- 172.21.15.19:0/324282005 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb9380a3bb0 0x7fb9380a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb948000c00 tx=0x7fb948031510 comp rx=0 tx=0).stop 2024-04-25T12:07:00.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.044+0000 7fb953aa7640 1 -- 172.21.15.19:0/324282005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c101fe0 msgr2=0x7fb94c103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.044+0000 7fb953aa7640 1 --2- 172.21.15.19:0/324282005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c101fe0 0x7fb94c103450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00b450 tx=0x7fb93c00b910 comp rx=0 tx=0).stop 2024-04-25T12:07:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.044+0000 7fb953aa7640 1 -- 172.21.15.19:0/324282005 shutdown_connections 2024-04-25T12:07:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.044+0000 7fb953aa7640 1 --2- 172.21.15.19:0/324282005 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb9380a3bb0 0x7fb9380a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.044+0000 7fb953aa7640 1 --2- 172.21.15.19:0/324282005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb94c1029a0 0x7fb94c103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.044+0000 7fb953aa7640 1 --2- 172.21.15.19:0/324282005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb94c101fe0 0x7fb94c103450 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.044+0000 7fb953aa7640 1 --2- 172.21.15.19:0/324282005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb94c100e90 0x7fb94c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.044+0000 7fb953aa7640 1 -- 172.21.15.19:0/324282005 >> 172.21.15.19:0/324282005 conn(0x7fb94c0fc400 msgr2=0x7fb94c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.044+0000 7fb953aa7640 1 -- 172.21.15.19:0/324282005 shutdown_connections 2024-04-25T12:07:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.044+0000 7fb953aa7640 1 -- 172.21.15.19:0/324282005 wait complete. 2024-04-25T12:07:00.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:59 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1204841546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:07:00.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:59 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/921415252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:07:00.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:59 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1041319795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:07:00.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:06:59 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1674470255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:07:00.054 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:07:00.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.158+0000 7f3ca61c4640 1 -- 172.21.15.19:0/580602396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca01029a0 msgr2=0x7f3ca01092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.158+0000 7f3ca61c4640 1 --2- 172.21.15.19:0/580602396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca01029a0 0x7f3ca01092c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009ca0 tx=0x7f3c9802a8c0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.158+0000 7f3ca61c4640 1 -- 172.21.15.19:0/580602396 shutdown_connections 2024-04-25T12:07:00.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.158+0000 7f3ca61c4640 1 --2- 172.21.15.19:0/580602396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca01029a0 0x7f3ca01092c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.158+0000 7f3ca61c4640 1 --2- 172.21.15.19:0/580602396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ca0101fe0 0x7f3ca0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.158+0000 7f3ca61c4640 1 --2- 172.21.15.19:0/580602396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca0100e90 0x7f3ca01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.158+0000 7f3ca61c4640 1 -- 172.21.15.19:0/580602396 >> 172.21.15.19:0/580602396 conn(0x7f3ca00fc400 msgr2=0x7f3ca00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.158+0000 7f3ca61c4640 1 -- 172.21.15.19:0/580602396 shutdown_connections 2024-04-25T12:07:00.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.158+0000 7f3ca61c4640 1 -- 172.21.15.19:0/580602396 wait complete. 2024-04-25T12:07:00.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.158+0000 7f3ca61c4640 1 Processor -- start 2024-04-25T12:07:00.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.158+0000 7f3ca61c4640 1 -- start start 2024-04-25T12:07:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3ca61c4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ca0100e90 0x7f3ca0104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3ca61c4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca0101fe0 0x7f3ca0103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3ca61c4640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca01029a0 0x7f3ca0103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3ca61c4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca010cc20 con 0x7f3ca0101fe0 2024-04-25T12:07:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3ca61c4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca010caa0 con 0x7f3ca0100e90 2024-04-25T12:07:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3ca61c4640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca010cda0 con 0x7f3ca01029a0 2024-04-25T12:07:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca0101fe0 0x7f3ca0103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca0101fe0 0x7f3ca0103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55362/0 (socket says 172.21.15.19:55362) 2024-04-25T12:07:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9effd640 1 -- 172.21.15.19:0/3739161921 learned_addr learned my addr 172.21.15.19:0/3739161921 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9ffff640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca01029a0 0x7f3ca0103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9f7fe640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ca0100e90 0x7f3ca0104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9effd640 1 -- 172.21.15.19:0/3739161921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca01029a0 msgr2=0x7f3ca0103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9effd640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca01029a0 0x7f3ca0103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9effd640 1 -- 172.21.15.19:0/3739161921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ca0100e90 msgr2=0x7f3ca0104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9effd640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ca0100e90 0x7f3ca0104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9effd640 1 -- 172.21.15.19:0/3739161921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca0104220 con 0x7f3ca0101fe0 2024-04-25T12:07:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9ffff640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca01029a0 0x7f3ca0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:07:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9effd640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca0101fe0 0x7f3ca0103450 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400b450 tx=0x7f3c9400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9cff9640 1 -- 172.21.15.19:0/3739161921 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94015020 con 0x7f3ca0101fe0 2024-04-25T12:07:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9cff9640 1 -- 172.21.15.19:0/3739161921 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3c94011d60 con 0x7f3ca0101fe0 2024-04-25T12:07:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3ca61c4640 1 -- 172.21.15.19:0/3739161921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca01a2510 con 0x7f3ca0101fe0 2024-04-25T12:07:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3c9cff9640 1 -- 172.21.15.19:0/3739161921 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9400f340 con 0x7f3ca0101fe0 2024-04-25T12:07:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.159+0000 7f3ca61c4640 1 -- 172.21.15.19:0/3739161921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca01a2ae0 con 0x7f3ca0101fe0 2024-04-25T12:07:00.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.160+0000 7f3ca61c4640 1 -- 172.21.15.19:0/3739161921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca01a2da0 con 0x7f3ca0101fe0 2024-04-25T12:07:00.163 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:59 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1204841546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:07:00.163 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:59 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/921415252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:07:00.163 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:59 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1041319795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:07:00.163 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:06:59 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1674470255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:07:00.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.162+0000 7f3c9cff9640 1 -- 172.21.15.19:0/3739161921 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3c94018020 con 0x7f3ca0101fe0 2024-04-25T12:07:00.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.162+0000 7f3c9cff9640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3c880a3bb0 0x7f3c880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.163+0000 7f3c9f7fe640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3c880a3bb0 0x7f3c880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.163+0000 7f3c9cff9640 1 -- 172.21.15.19:0/3739161921 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f3c940cffa0 con 0x7f3ca0101fe0 2024-04-25T12:07:00.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.163+0000 7f3c9f7fe640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3c880a3bb0 0x7f3c880a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009890 tx=0x7f3c90008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:59 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1204841546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:07:00.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:59 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/921415252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:07:00.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:59 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1041319795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:07:00.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:06:59 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1674470255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:07:00.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.167+0000 7f3c9cff9640 1 -- 172.21.15.19:0/3739161921 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c9401f020 con 0x7f3ca0101fe0 2024-04-25T12:07:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.357+0000 7f3ca61c4640 1 -- 172.21.15.19:0/3739161921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ca01a3470 con 0x7f3ca0101fe0 2024-04-25T12:07:00.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.358+0000 7f3c9cff9640 1 -- 172.21.15.19:0/3739161921 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22731 (secure 0 0 0) 0x7f3c9400d040 con 0x7f3ca0101fe0 2024-04-25T12:07:00.357 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:07:00.357 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":87,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:05:55.975412+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:05:01.658354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:05:02.969078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.360+0000 7f3ca61c4640 1 -- 172.21.15.19:0/3739161921 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3c880a3bb0 msgr2=0x7f3c880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.360+0000 7f3ca61c4640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3c880a3bb0 0x7f3c880a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009890 tx=0x7f3c90008040 comp rx=0 tx=0).stop 2024-04-25T12:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.360+0000 7f3ca61c4640 1 -- 172.21.15.19:0/3739161921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca0101fe0 msgr2=0x7f3ca0103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.360+0000 7f3ca61c4640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca0101fe0 0x7f3ca0103450 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400b450 tx=0x7f3c9400b910 comp rx=0 tx=0).stop 2024-04-25T12:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.360+0000 7f3ca61c4640 1 -- 172.21.15.19:0/3739161921 shutdown_connections 2024-04-25T12:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.360+0000 7f3ca61c4640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f3c880a3bb0 0x7f3c880a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.360+0000 7f3ca61c4640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ca01029a0 0x7f3ca0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.360+0000 7f3ca61c4640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca0101fe0 0x7f3ca0103450 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.360+0000 7f3ca61c4640 1 --2- 172.21.15.19:0/3739161921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ca0100e90 0x7f3ca0104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.360+0000 7f3ca61c4640 1 -- 172.21.15.19:0/3739161921 >> 172.21.15.19:0/3739161921 conn(0x7f3ca00fc400 msgr2=0x7f3ca00fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.360+0000 7f3ca61c4640 1 -- 172.21.15.19:0/3739161921 shutdown_connections 2024-04-25T12:07:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.360+0000 7f3ca61c4640 1 -- 172.21.15.19:0/3739161921 wait complete. 2024-04-25T12:07:00.371 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T12:05:04.008679+0000', 'modified': '2024-04-25T12:05:11.186222+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24514, 'mds_1': 24500, 'mds_2': 24496, 'mds_3': 24484, 'mds_4': 14655}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14643': {'gid': 14643, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/1694897630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 1694897630}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 1694897630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14655': {'gid': 14655, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.19:6841/3247157172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 3247157172}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 3247157172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6833/4176941259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6832', 'nonce': 4176941259}, {'type': 'v1', 'addr': '172.21.15.150:6833', 'nonce': 4176941259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.169:6833/2217300694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6832', 'nonce': 2217300694}, {'type': 'v1', 'addr': '172.21.15.169:6833', 'nonce': 2217300694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6835/1405540140', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6834', 'nonce': 1405540140}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 1405540140}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6837/3918801450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 3918801450}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 3918801450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.169:6835/3316630328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6834', 'nonce': 3316630328}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 3316630328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.169:6837/1253767842', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6836', 'nonce': 1253767842}, {'type': 'v1', 'addr': '172.21.15.169:6837', 'nonce': 1253767842}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.150:6839/2553528303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 2553528303}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 2553528303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.169:6839/1723439500', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6838', 'nonce': 1723439500}, {'type': 'v1', 'addr': '172.21.15.169:6839', 'nonce': 1723439500}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24514, 'qdb_cluster': [24500, 24514, 24496, 24484, 14655]} 2024-04-25T12:07:00.371 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T12:07:00.371 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T12:07:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.476+0000 7f6ce6e05640 1 -- 172.21.15.19:0/832753904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ce0103b70 msgr2=0x7f6ce010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.476+0000 7f6ce6e05640 1 --2- 172.21.15.19:0/832753904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ce0103b70 0x7f6ce010a400 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009ca0 tx=0x7f6cd802a8c0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.476+0000 7f6ce6e05640 1 -- 172.21.15.19:0/832753904 shutdown_connections 2024-04-25T12:07:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.476+0000 7f6ce6e05640 1 --2- 172.21.15.19:0/832753904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ce0103b70 0x7f6ce010a400 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.476+0000 7f6ce6e05640 1 --2- 172.21.15.19:0/832753904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ce01031b0 0x7f6ce0103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.476+0000 7f6ce6e05640 1 --2- 172.21.15.19:0/832753904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce0101fc0 0x7f6ce01023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.476+0000 7f6ce6e05640 1 -- 172.21.15.19:0/832753904 >> 172.21.15.19:0/832753904 conn(0x7f6ce00fd530 msgr2=0x7f6ce00ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.476+0000 7f6ce6e05640 1 -- 172.21.15.19:0/832753904 shutdown_connections 2024-04-25T12:07:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.476+0000 7f6ce6e05640 1 -- 172.21.15.19:0/832753904 wait complete. 2024-04-25T12:07:00.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.476+0000 7f6ce6e05640 1 Processor -- start 2024-04-25T12:07:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6ce6e05640 1 -- start start 2024-04-25T12:07:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6ce6e05640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce0101fc0 0x7f6ce0198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6ce6e05640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ce01031b0 0x7f6ce0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6ce6e05640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ce0103b70 0x7f6ce019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6ce6e05640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce010dc20 con 0x7f6ce01031b0 2024-04-25T12:07:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6ce6e05640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce010daa0 con 0x7f6ce0103b70 2024-04-25T12:07:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6ce6e05640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce010dda0 con 0x7f6ce0101fc0 2024-04-25T12:07:00.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6cd7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ce01031b0 0x7f6ce0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6cd7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ce01031b0 0x7f6ce0198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55386/0 (socket says 172.21.15.19:55386) 2024-04-25T12:07:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6cd7fff640 1 -- 172.21.15.19:0/1182418250 learned_addr learned my addr 172.21.15.19:0/1182418250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6cd7fff640 1 -- 172.21.15.19:0/1182418250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce0101fc0 msgr2=0x7f6ce0198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6ce4ba1640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce0101fc0 0x7f6ce0198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6ce53a2640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ce0103b70 0x7f6ce019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce0101fc0 0x7f6ce0198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6cd7fff640 1 -- 172.21.15.19:0/1182418250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ce0103b70 msgr2=0x7f6ce019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ce0103b70 0x7f6ce019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6cd7fff640 1 -- 172.21.15.19:0/1182418250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce019fe60 con 0x7f6ce01031b0 2024-04-25T12:07:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6ce4ba1640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce0101fc0 0x7f6ce0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:07:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6ce53a2640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ce0103b70 0x7f6ce019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:07:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ce01031b0 0x7f6ce0198780 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000d950 tx=0x7f6cd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6cd5ffb640 1 -- 172.21.15.19:0/1182418250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0017070 con 0x7f6ce01031b0 2024-04-25T12:07:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6cd5ffb640 1 -- 172.21.15.19:0/1182418250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6cd0009c30 con 0x7f6ce01031b0 2024-04-25T12:07:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6ce6e05640 1 -- 172.21.15.19:0/1182418250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce01a0150 con 0x7f6ce01031b0 2024-04-25T12:07:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.477+0000 7f6cd5ffb640 1 -- 172.21.15.19:0/1182418250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd000f6b0 con 0x7f6ce01031b0 2024-04-25T12:07:00.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.478+0000 7f6ce6e05640 1 -- 172.21.15.19:0/1182418250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce01a06a0 con 0x7f6ce01031b0 2024-04-25T12:07:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.480+0000 7f6cd5ffb640 1 -- 172.21.15.19:0/1182418250 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6cd00109c0 con 0x7f6ce01031b0 2024-04-25T12:07:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.480+0000 7f6ce9bf7740 1 Processor -- start 2024-04-25T12:07:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.480+0000 7f6ce9bf7740 1 -- start start 2024-04-25T12:07:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.480+0000 7f6cd5ffb640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6cc80a3bb0 0x7f6cc80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.480+0000 7f6ce9bf7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e55b80 0x555c67e55fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.480+0000 7f6ce9bf7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e56d50 0x555c67e571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.480+0000 7f6ce9bf7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e57710 0x555c67e5dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.480+0000 7f6ce9bf7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555c67dbac50 con 0x555c67e56d50 2024-04-25T12:07:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.480+0000 7f6ce9bf7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555c67d57830 con 0x555c67e57710 2024-04-25T12:07:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.480+0000 7f6ce9bf7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555c67db6c10 con 0x555c67e55b80 2024-04-25T12:07:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.480+0000 7f6ce53a2640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6cc80a3bb0 0x7f6cc80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6cd7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e56d50 0x555c67e571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6cd7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e56d50 0x555c67e571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55394/0 (socket says 172.21.15.19:55394) 2024-04-25T12:07:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6cd7fff640 1 -- 172.21.15.19:0/672922895 learned_addr learned my addr 172.21.15.19:0/672922895 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6cd5ffb640 1 -- 172.21.15.19:0/1182418250 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f6cd00d0100 con 0x7f6ce01031b0 2024-04-25T12:07:00.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6ce4ba1640 1 --2- 172.21.15.19:0/672922895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e55b80 0x555c67e55fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6ce53a2640 1 --2- 172.21.15.19:0/672922895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e57710 0x555c67e5dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6cd7fff640 1 -- 172.21.15.19:0/672922895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e55b80 msgr2=0x555c67e55fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/672922895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e55b80 0x555c67e55fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6cd7fff640 1 -- 172.21.15.19:0/672922895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e57710 msgr2=0x555c67e5dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/672922895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e57710 0x555c67e5dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6cd7fff640 1 -- 172.21.15.19:0/672922895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c67acd2d0 con 0x555c67e56d50 2024-04-25T12:07:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/672922895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e56d50 0x555c67e571d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6cd009c350 tx=0x7f6cd0010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26903db4f573425 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6ce53a2640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6cc80a3bb0 0x7f6cc80a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009ca0 tx=0x7f6cd8031450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6cc6ffd640 1 -- 172.21.15.19:0/672922895 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0020070 con 0x555c67e56d50 2024-04-25T12:07:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6cc6ffd640 1 -- 172.21.15.19:0/672922895 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6cd0009de0 con 0x555c67e56d50 2024-04-25T12:07:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.481+0000 7f6cc6ffd640 1 -- 172.21.15.19:0/672922895 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd009a6c0 con 0x555c67e56d50 2024-04-25T12:07:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.482+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/672922895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e56d50 msgr2=0x555c67e571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.482+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/672922895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e56d50 0x555c67e571d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6cd009c350 tx=0x7f6cd0010d60 comp rx=0 tx=0).stop 2024-04-25T12:07:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.482+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/672922895 shutdown_connections 2024-04-25T12:07:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.482+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/672922895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e57710 0x555c67e5dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.482+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/672922895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e56d50 0x555c67e571d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.482+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/672922895 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e55b80 0x555c67e55fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.482+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/672922895 >> 172.21.15.19:0/672922895 conn(0x555c67e51070 msgr2=0x555c67e51470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.482+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/672922895 shutdown_connections 2024-04-25T12:07:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.482+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/672922895 wait complete. 2024-04-25T12:07:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.482+0000 7f6ce9bf7740 1 -- start start 2024-04-25T12:07:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.482+0000 7f6ce9bf7740 1 Processor -- start 2024-04-25T12:07:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce9bf7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e55b80 0x555c67e600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce9bf7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e56d50 0x555c67e60600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce9bf7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e57710 0x555c67e675d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce9bf7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555c67e5e720 con 0x555c67e55b80 2024-04-25T12:07:00.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce9bf7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555c67d55470 con 0x555c67e57710 2024-04-25T12:07:00.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce9bf7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555c67e5e8a0 con 0x555c67e56d50 2024-04-25T12:07:00.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce9bf7740 10 client.0 _ll_register_callbacks cb 0x555c67d7af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:07:00.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce4ba1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e55b80 0x555c67e600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce4ba1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e55b80 0x555c67e600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55400/0 (socket says 172.21.15.19:55400) 2024-04-25T12:07:00.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce4ba1640 1 -- 172.21.15.19:0/189926926 learned_addr learned my addr 172.21.15.19:0/189926926 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce53a2640 1 --2- 172.21.15.19:0/189926926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e57710 0x555c67e675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/189926926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e56d50 0x555c67e60600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce4ba1640 1 -- 172.21.15.19:0/189926926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e56d50 msgr2=0x555c67e60600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce4ba1640 1 --2- 172.21.15.19:0/189926926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e56d50 0x555c67e60600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce4ba1640 1 -- 172.21.15.19:0/189926926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e57710 msgr2=0x555c67e675d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce4ba1640 1 --2- 172.21.15.19:0/189926926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e57710 0x555c67e675d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce4ba1640 1 -- 172.21.15.19:0/189926926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c67b976b0 con 0x555c67e55b80 2024-04-25T12:07:00.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/189926926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e56d50 0x555c67e60600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:07:00.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce4ba1640 1 --2- 172.21.15.19:0/189926926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e55b80 0x555c67e600c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc010710 tx=0x7f6cdc010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397686bb4335a2f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6cbffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:07:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6cbffff640 1 -- 172.21.15.19:0/189926926 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc01d060 con 0x555c67e55b80 2024-04-25T12:07:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6cbffff640 1 -- 172.21.15.19:0/189926926 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f6cdc00ed40 con 0x555c67e55b80 2024-04-25T12:07:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.483+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/189926926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cd0009de0 con 0x555c67e55b80 2024-04-25T12:07:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.484+0000 7f6cbffff640 1 -- 172.21.15.19:0/189926926 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6cdc018430 con 0x555c67e55b80 2024-04-25T12:07:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.484+0000 7f6ce9bf7740 10 client.15279 fetch_fsmap learned FSMap version 26 2024-04-25T12:07:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.484+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/189926926 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555c67e6aa10 con 0x555c67e55b80 2024-04-25T12:07:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.484+0000 7f6cbffff640 1 -- 172.21.15.19:0/189926926 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6cd0009de0 con 0x555c67e55b80 2024-04-25T12:07:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.484+0000 7f6cbffff640 1 -- 172.21.15.19:0/189926926 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6cdc022590 con 0x555c67e55b80 2024-04-25T12:07:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.484+0000 7f6ce9bf7740 10 client.15279 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:07:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.484+0000 7f6ce9bf7740 10 client.15279 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T12:07:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.484+0000 7f6ce9bf7740 20 client.15279 populate_metadata read hostname 'smithi019' 2024-04-25T12:07:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.484+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/189926926 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x555c67e6d0b0 0x555c67e6e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.484+0000 7f6ce9bf7740 4 client.15279 mds_command: new command op to 14655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T12:07:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.484+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/189926926 --> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c67dc8670 con 0x555c67e6d0b0 2024-04-25T12:07:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.484+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/189926926 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x555c67e6d0b0 0x555c67e6e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.484+0000 7f6cbffff640 10 client.15279 ms_handle_connect on v2:172.21.15.19:6840/3247157172 2024-04-25T12:07:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.484+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/189926926 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x555c67e6d0b0 0x555c67e6e520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2531b86a31be3617 server_cookie=764e2875314d9410 in_seq=0 out_seq=0 2024-04-25T12:07:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6cbffff640 1 -- 172.21.15.19:0/189926926 <== mds.4 v2:172.21.15.19:6840/3247157172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c67dc8670 con 0x555c67e6d0b0 2024-04-25T12:07:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6cbffff640 10 client.15279 handle_command_reply: tid=0 2024-04-25T12:07:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6ce9bf7740 1 client.15279 shutdown 2024-04-25T12:07:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/189926926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e55b80 msgr2=0x555c67e600c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/189926926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e55b80 0x555c67e600c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc010710 tx=0x7f6cdc010bd0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/189926926 shutdown_connections 2024-04-25T12:07:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/189926926 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x555c67e6d0b0 0x555c67e6e520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/189926926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e57710 0x555c67e675d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/189926926 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e56d50 0x555c67e60600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6cbffff640 0 client.15279 ms_handle_reset on v2:172.21.15.19:6840/3247157172 2024-04-25T12:07:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/189926926 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e55b80 0x555c67e600c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/189926926 >> 172.21.15.19:0/189926926 conn(0x555c67e51070 msgr2=0x555c67e501d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/189926926 shutdown_connections 2024-04-25T12:07:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/189926926 wait complete. 2024-04-25T12:07:00.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.485+0000 7f6ce9bf7740 20 client.15279 trim_cache size 0 max 16384 2024-04-25T12:07:00.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.504+0000 7f6ce9bf7740 1 Processor -- start 2024-04-25T12:07:00.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.504+0000 7f6ce9bf7740 1 -- start start 2024-04-25T12:07:00.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.504+0000 7f6ce9bf7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e55b80 0x555c67e51450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.504+0000 7f6ce9bf7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e56d50 0x555c67e67590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.504+0000 7f6ce9bf7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e57710 0x555c67e54660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.504+0000 7f6ce9bf7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555c67e6c860 con 0x555c67e56d50 2024-04-25T12:07:00.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.504+0000 7f6ce9bf7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555c67e6cf30 con 0x555c67e55b80 2024-04-25T12:07:00.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.504+0000 7f6ce9bf7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555c67e6d600 con 0x555c67e57710 2024-04-25T12:07:00.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.504+0000 7f6cd7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e56d50 0x555c67e67590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.504+0000 7f6cd7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e56d50 0x555c67e67590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55414/0 (socket says 172.21.15.19:55414) 2024-04-25T12:07:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.504+0000 7f6cd7fff640 1 -- 172.21.15.19:0/1245777397 learned_addr learned my addr 172.21.15.19:0/1245777397 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.505+0000 7f6cd7fff640 1 -- 172.21.15.19:0/1245777397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e57710 msgr2=0x555c67e54660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.505+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/1245777397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e57710 0x555c67e54660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.505+0000 7f6cd7fff640 1 -- 172.21.15.19:0/1245777397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e55b80 msgr2=0x555c67e51450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.505+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/1245777397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e55b80 0x555c67e51450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.505+0000 7f6cd7fff640 1 -- 172.21.15.19:0/1245777397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c67e51d40 con 0x555c67e56d50 2024-04-25T12:07:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.505+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/1245777397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e56d50 0x555c67e67590 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6cd009c350 tx=0x7f6cd00a0770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c7b2ea44492a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.505+0000 7f6cc67fc640 1 -- 172.21.15.19:0/1245777397 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd00b0e60 con 0x555c67e56d50 2024-04-25T12:07:00.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.505+0000 7f6cc67fc640 1 -- 172.21.15.19:0/1245777397 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6cd0009de0 con 0x555c67e56d50 2024-04-25T12:07:00.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.505+0000 7f6cc67fc640 1 -- 172.21.15.19:0/1245777397 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0020040 con 0x555c67e56d50 2024-04-25T12:07:00.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/1245777397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e56d50 msgr2=0x555c67e67590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/1245777397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e56d50 0x555c67e67590 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6cd009c350 tx=0x7f6cd00a0770 comp rx=0 tx=0).stop 2024-04-25T12:07:00.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/1245777397 shutdown_connections 2024-04-25T12:07:00.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/1245777397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e57710 0x555c67e54660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/1245777397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e56d50 0x555c67e67590 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/1245777397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e55b80 0x555c67e51450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/1245777397 >> 172.21.15.19:0/1245777397 conn(0x555c67e51070 msgr2=0x555c67e53f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/1245777397 shutdown_connections 2024-04-25T12:07:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/1245777397 wait complete. 2024-04-25T12:07:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 -- start start 2024-04-25T12:07:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 Processor -- start 2024-04-25T12:07:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e55b80 0x555c67e62c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e56d50 0x555c67e631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e57710 0x555c67e661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555c67e6e3a0 con 0x555c67e55b80 2024-04-25T12:07:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555c67e6dcd0 con 0x555c67e56d50 2024-04-25T12:07:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x555c67e6ea70 con 0x555c67e57710 2024-04-25T12:07:00.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce9bf7740 10 client.0 _ll_register_callbacks cb 0x7f6cdc00b660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:07:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce4ba1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e55b80 0x555c67e62c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce4ba1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e55b80 0x555c67e62c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55424/0 (socket says 172.21.15.19:55424) 2024-04-25T12:07:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.506+0000 7f6ce4ba1640 1 -- 172.21.15.19:0/1857574034 learned_addr learned my addr 172.21.15.19:0/1857574034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e56d50 0x555c67e631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6ce53a2640 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e57710 0x555c67e661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6ce4ba1640 1 -- 172.21.15.19:0/1857574034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e57710 msgr2=0x555c67e661d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6ce4ba1640 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e57710 0x555c67e661d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6ce4ba1640 1 -- 172.21.15.19:0/1857574034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e56d50 msgr2=0x555c67e631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6ce4ba1640 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e56d50 0x555c67e631b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6ce4ba1640 1 -- 172.21.15.19:0/1857574034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c67e667c0 con 0x555c67e55b80 2024-04-25T12:07:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e56d50 0x555c67e631b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:07:00.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6ce53a2640 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e57710 0x555c67e661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:07:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6ce4ba1640 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e55b80 0x555c67e62c70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc022bf0 tx=0x7f6cdc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a655c97dfaea703b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6cbffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:07:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6cbffff640 1 -- 172.21.15.19:0/1857574034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0227c0 con 0x555c67e55b80 2024-04-25T12:07:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6cbffff640 1 -- 172.21.15.19:0/1857574034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f6cdc005030 con 0x555c67e55b80 2024-04-25T12:07:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/1857574034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cd0009de0 con 0x555c67e55b80 2024-04-25T12:07:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6cbffff640 1 -- 172.21.15.19:0/1857574034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6cdc01d020 con 0x555c67e55b80 2024-04-25T12:07:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6ce9bf7740 10 client.15285 fetch_fsmap learned FSMap version 26 2024-04-25T12:07:00.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/1857574034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555c67ed30d0 con 0x555c67e55b80 2024-04-25T12:07:00.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6cbffff640 1 -- 172.21.15.19:0/1857574034 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6cd0009de0 con 0x555c67e55b80 2024-04-25T12:07:00.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.507+0000 7f6cbffff640 1 -- 172.21.15.19:0/1857574034 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6cdc004b60 con 0x555c67e55b80 2024-04-25T12:07:00.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 10 client.15285 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:07:00.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 10 client.15285 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T12:07:00.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 20 client.15285 populate_metadata read hostname 'smithi019' 2024-04-25T12:07:00.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x555c67ed7c80 0x555c67ed6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 4 client.15285 mds_command: new command op to 14655 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T12:07:00.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/1857574034 --> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] -- command(tid 0: {"prefix": "status"}) -- 0x555c67dc24e0 con 0x555c67ed7c80 2024-04-25T12:07:00.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x555c67ed7c80 0x555c67ed6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6cbffff640 10 client.15285 ms_handle_connect on v2:172.21.15.19:6840/3247157172 2024-04-25T12:07:00.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6cd7fff640 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x555c67ed7c80 0x555c67ed6990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=54569ac5323d2ab3 server_cookie=509dcdf50e03411c in_seq=0 out_seq=0 2024-04-25T12:07:00.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6cbffff640 1 -- 172.21.15.19:0/1857574034 <== mds.4 v2:172.21.15.19:6840/3247157172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x555c67dc24e0 con 0x555c67ed7c80 2024-04-25T12:07:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6cbffff640 10 client.15285 handle_command_reply: tid=0 2024-04-25T12:07:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 1 client.15285 shutdown 2024-04-25T12:07:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/1857574034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e55b80 msgr2=0x555c67e62c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e55b80 0x555c67e62c70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc022bf0 tx=0x7f6cdc00ed70 comp rx=0 tx=0).stop 2024-04-25T12:07:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/1857574034 shutdown_connections 2024-04-25T12:07:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.19:6840/3247157172,v1:172.21.15.19:6841/3247157172] conn(0x555c67ed7c80 0x555c67ed6990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555c67e57710 0x555c67e661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555c67e56d50 0x555c67e631b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 1 --2- 172.21.15.19:0/1857574034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555c67e55b80 0x555c67e62c70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6cbffff640 0 client.15285 ms_handle_reset on v2:172.21.15.19:6840/3247157172 2024-04-25T12:07:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/1857574034 >> 172.21.15.19:0/1857574034 conn(0x555c67e51070 msgr2=0x555c67e501d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/1857574034 shutdown_connections 2024-04-25T12:07:00.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.508+0000 7f6ce9bf7740 1 -- 172.21.15.19:0/1857574034 wait complete. 2024-04-25T12:07:00.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.509+0000 7f6ce9bf7740 20 client.15285 trim_cache size 0 max 16384 2024-04-25T12:07:00.516 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:07:00.516 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "be62e290-02fa-11ef-bc93-c7b262605968", 2024-04-25T12:07:00.516 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-04-25T12:07:00.516 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 14655, 2024-04-25T12:07:00.516 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T12:07:00.516 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T12:07:00.517 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T12:07:00.517 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 110.32639698, 2024-04-25T12:07:00.517 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-25T12:07:00.517 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 87, 2024-04-25T12:07:00.517 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T12:07:00.517 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 125.59247223200001 2024-04-25T12:07:00.517 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:07:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.509+0000 7f6ce6e05640 1 -- 172.21.15.19:0/1182418250 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6cc80a3bb0 msgr2=0x7f6cc80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.509+0000 7f6ce6e05640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6cc80a3bb0 0x7f6cc80a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009ca0 tx=0x7f6cd8031450 comp rx=0 tx=0).stop 2024-04-25T12:07:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.509+0000 7f6ce6e05640 1 -- 172.21.15.19:0/1182418250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ce01031b0 msgr2=0x7f6ce0198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.509+0000 7f6ce6e05640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ce01031b0 0x7f6ce0198780 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000d950 tx=0x7f6cd000de10 comp rx=0 tx=0).stop 2024-04-25T12:07:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.509+0000 7f6ce6e05640 1 -- 172.21.15.19:0/1182418250 shutdown_connections 2024-04-25T12:07:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.509+0000 7f6ce6e05640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f6cc80a3bb0 0x7f6cc80a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.509+0000 7f6ce6e05640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ce0103b70 0x7f6ce019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.509+0000 7f6ce6e05640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ce01031b0 0x7f6ce0198780 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.509+0000 7f6ce6e05640 1 --2- 172.21.15.19:0/1182418250 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ce0101fc0 0x7f6ce0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.509+0000 7f6ce6e05640 1 -- 172.21.15.19:0/1182418250 >> 172.21.15.19:0/1182418250 conn(0x7f6ce00fd530 msgr2=0x7f6ce01064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.509+0000 7f6ce6e05640 1 -- 172.21.15.19:0/1182418250 shutdown_connections 2024-04-25T12:07:00.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.509+0000 7f6ce6e05640 1 -- 172.21.15.19:0/1182418250 wait complete. 2024-04-25T12:07:00.520 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T12:07:00.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.624+0000 7f4428f36640 1 -- 172.21.15.19:0/2214793770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44241029a0 msgr2=0x7f44241092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.624+0000 7f4428f36640 1 --2- 172.21.15.19:0/2214793770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44241029a0 0x7f44241092c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f441c009ca0 tx=0x7f441c02a8c0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.625+0000 7f4428f36640 1 -- 172.21.15.19:0/2214793770 shutdown_connections 2024-04-25T12:07:00.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.625+0000 7f4428f36640 1 --2- 172.21.15.19:0/2214793770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44241029a0 0x7f44241092c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.625+0000 7f4428f36640 1 --2- 172.21.15.19:0/2214793770 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4424101fe0 0x7f4424102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.625+0000 7f4428f36640 1 --2- 172.21.15.19:0/2214793770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4424100e90 0x7f44241012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.625+0000 7f4428f36640 1 -- 172.21.15.19:0/2214793770 >> 172.21.15.19:0/2214793770 conn(0x7f44240fc400 msgr2=0x7f44240fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.625+0000 7f4428f36640 1 -- 172.21.15.19:0/2214793770 shutdown_connections 2024-04-25T12:07:00.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.625+0000 7f4428f36640 1 -- 172.21.15.19:0/2214793770 wait complete. 2024-04-25T12:07:00.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.625+0000 7f4428f36640 1 Processor -- start 2024-04-25T12:07:00.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.625+0000 7f4428f36640 1 -- start start 2024-04-25T12:07:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.625+0000 7f4428f36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424100e90 0x7f4424198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f4428f36640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4424101fe0 0x7f44241986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f4428f36640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44241029a0 0x7f442419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f4428f36640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f442410cb10 con 0x7f4424100e90 2024-04-25T12:07:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f4428f36640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f442410c990 con 0x7f44241029a0 2024-04-25T12:07:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f4428f36640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f442410cc90 con 0x7f4424101fe0 2024-04-25T12:07:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f442259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424100e90 0x7f4424198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f442259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424100e90 0x7f4424198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55442/0 (socket says 172.21.15.19:55442) 2024-04-25T12:07:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f442259c640 1 -- 172.21.15.19:0/1857023117 learned_addr learned my addr 172.21.15.19:0/1857023117 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f442259c640 1 -- 172.21.15.19:0/1857023117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4424101fe0 msgr2=0x7f44241986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f442259c640 1 --2- 172.21.15.19:0/1857023117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4424101fe0 0x7f44241986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f442259c640 1 -- 172.21.15.19:0/1857023117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44241029a0 msgr2=0x7f442419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f442259c640 1 --2- 172.21.15.19:0/1857023117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44241029a0 0x7f442419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f442259c640 1 -- 172.21.15.19:0/1857023117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442419fdb0 con 0x7f4424100e90 2024-04-25T12:07:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f442259c640 1 --2- 172.21.15.19:0/1857023117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424100e90 0x7f4424198190 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f441400d950 tx=0x7f441400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f44137fe640 1 -- 172.21.15.19:0/1857023117 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414017070 con 0x7f4424100e90 2024-04-25T12:07:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f44137fe640 1 -- 172.21.15.19:0/1857023117 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4414009c30 con 0x7f4424100e90 2024-04-25T12:07:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f44137fe640 1 -- 172.21.15.19:0/1857023117 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441400f6b0 con 0x7f4424100e90 2024-04-25T12:07:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f4428f36640 1 -- 172.21.15.19:0/1857023117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44241a00a0 con 0x7f4424100e90 2024-04-25T12:07:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.626+0000 7f4428f36640 1 -- 172.21.15.19:0/1857023117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44241a0640 con 0x7f4424100e90 2024-04-25T12:07:00.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.628+0000 7f44137fe640 1 -- 172.21.15.19:0/1857023117 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f44140109c0 con 0x7f4424100e90 2024-04-25T12:07:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f442bc45740 1 Processor -- start 2024-04-25T12:07:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f442bc45740 1 -- start start 2024-04-25T12:07:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f442bc45740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92207f90 0x55bd922083b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f442bc45740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92209160 0x55bd922095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f442bc45740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209b20 0x55bd922103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f442bc45740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bd92168c50 con 0x55bd92209b20 2024-04-25T12:07:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f442bc45740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bd92105830 con 0x55bd92209160 2024-04-25T12:07:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f442bc45740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bd92164c10 con 0x55bd92207f90 2024-04-25T12:07:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f44137fe640 1 --2- 172.21.15.19:0/1857023117 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f440c0a3c30 0x7f440c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f4421d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209b20 0x55bd922103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f4421d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209b20 0x55bd922103b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55446/0 (socket says 172.21.15.19:55446) 2024-04-25T12:07:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f4421d9b640 1 -- 172.21.15.19:0/3315393772 learned_addr learned my addr 172.21.15.19:0/3315393772 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f4422d9d640 1 --2- 172.21.15.19:0/3315393772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92207f90 0x55bd922083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f442259c640 1 --2- 172.21.15.19:0/3315393772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92209160 0x55bd922095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f4422d9d640 1 --2- 172.21.15.19:0/1857023117 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f440c0a3c30 0x7f440c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f44137fe640 1 -- 172.21.15.19:0/1857023117 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f44140d0040 con 0x7f4424100e90 2024-04-25T12:07:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f4421d9b640 1 -- 172.21.15.19:0/3315393772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92207f90 msgr2=0x55bd922083b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.629+0000 7f4421d9b640 1 --2- 172.21.15.19:0/3315393772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92207f90 0x55bd922083b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.630+0000 7f4421d9b640 1 -- 172.21.15.19:0/3315393772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92209160 msgr2=0x55bd922095e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.630+0000 7f4421d9b640 1 --2- 172.21.15.19:0/3315393772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92209160 0x55bd922095e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.630+0000 7f4421d9b640 1 -- 172.21.15.19:0/3315393772 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd91e7b2d0 con 0x55bd92209b20 2024-04-25T12:07:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.630+0000 7f4421d9b640 1 --2- 172.21.15.19:0/3315393772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209b20 0x55bd922103b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f441800aa50 tx=0x7f441800af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e2cb94a3be8ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.630+0000 7f4422d9d640 1 --2- 172.21.15.19:0/1857023117 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f440c0a3c30 0x7f440c0a60f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f441c031a70 tx=0x7f441c0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.630+0000 7f4410ff9640 1 -- 172.21.15.19:0/3315393772 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441800ce20 con 0x55bd92209b20 2024-04-25T12:07:00.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.630+0000 7f4410ff9640 1 -- 172.21.15.19:0/3315393772 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f44180035f0 con 0x55bd92209b20 2024-04-25T12:07:00.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.630+0000 7f4410ff9640 1 -- 172.21.15.19:0/3315393772 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441800f720 con 0x55bd92209b20 2024-04-25T12:07:00.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 -- 172.21.15.19:0/3315393772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209b20 msgr2=0x55bd922103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 --2- 172.21.15.19:0/3315393772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209b20 0x55bd922103b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f441800aa50 tx=0x7f441800af10 comp rx=0 tx=0).stop 2024-04-25T12:07:00.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 -- 172.21.15.19:0/3315393772 shutdown_connections 2024-04-25T12:07:00.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 --2- 172.21.15.19:0/3315393772 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209b20 0x55bd922103b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 --2- 172.21.15.19:0/3315393772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92209160 0x55bd922095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 --2- 172.21.15.19:0/3315393772 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92207f90 0x55bd922083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 -- 172.21.15.19:0/3315393772 >> 172.21.15.19:0/3315393772 conn(0x55bd921ff070 msgr2=0x55bd921ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 -- 172.21.15.19:0/3315393772 shutdown_connections 2024-04-25T12:07:00.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 -- 172.21.15.19:0/3315393772 wait complete. 2024-04-25T12:07:00.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 -- start start 2024-04-25T12:07:00.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 Processor -- start 2024-04-25T12:07:00.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92207f90 0x55bd922123e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209160 0x55bd92212920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92209b20 0x55bd92219960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bd92210b30 con 0x55bd92209160 2024-04-25T12:07:00.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bd92103470 con 0x55bd92209b20 2024-04-25T12:07:00.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bd92210cb0 con 0x55bd92207f90 2024-04-25T12:07:00.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f442bc45740 10 client.0 _ll_register_callbacks cb 0x55bd92128f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:07:00.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f4421d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209160 0x55bd92212920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f4421d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209160 0x55bd92212920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55462/0 (socket says 172.21.15.19:55462) 2024-04-25T12:07:00.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.631+0000 7f4421d9b640 1 -- 172.21.15.19:0/1539728641 learned_addr learned my addr 172.21.15.19:0/1539728641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f442259c640 1 --2- 172.21.15.19:0/1539728641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92207f90 0x55bd922123e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f4421d9b640 1 -- 172.21.15.19:0/1539728641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92207f90 msgr2=0x55bd922123e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f4421d9b640 1 --2- 172.21.15.19:0/1539728641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92207f90 0x55bd922123e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f4421d9b640 1 -- 172.21.15.19:0/1539728641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92209b20 msgr2=0x55bd92219960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f4421d9b640 1 --2- 172.21.15.19:0/1539728641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92209b20 0x55bd92219960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f4421d9b640 1 -- 172.21.15.19:0/1539728641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd91f456b0 con 0x55bd92209160 2024-04-25T12:07:00.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f442259c640 1 --2- 172.21.15.19:0/1539728641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92207f90 0x55bd922123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:07:00.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f4421d9b640 1 --2- 172.21.15.19:0/1539728641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209160 0x55bd92212920 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4418004700 tx=0x7f4418004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cc41b03aac20e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f4409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:07:00.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f4409ffb640 1 -- 172.21.15.19:0/1539728641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418028060 con 0x55bd92209160 2024-04-25T12:07:00.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f4409ffb640 1 -- 172.21.15.19:0/1539728641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f44180039b0 con 0x55bd92209160 2024-04-25T12:07:00.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f442bc45740 1 -- 172.21.15.19:0/1539728641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44180035f0 con 0x55bd92209160 2024-04-25T12:07:00.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f4409ffb640 1 -- 172.21.15.19:0/1539728641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4418004340 con 0x55bd92209160 2024-04-25T12:07:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f442bc45740 10 client.15297 fetch_fsmap learned FSMap version 26 2024-04-25T12:07:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f442bc45740 1 -- 172.21.15.19:0/1539728641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd9221ccd0 con 0x55bd92209160 2024-04-25T12:07:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f4409ffb640 1 -- 172.21.15.19:0/1539728641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f44180035f0 con 0x55bd92209160 2024-04-25T12:07:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.632+0000 7f4409ffb640 1 -- 172.21.15.19:0/1539728641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f441802e400 con 0x55bd92209160 2024-04-25T12:07:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.633+0000 7f442bc45740 10 client.15297 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:07:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.633+0000 7f442bc45740 10 client.15297 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T12:07:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.633+0000 7f442bc45740 20 client.15297 populate_metadata read hostname 'smithi019' 2024-04-25T12:07:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.633+0000 7f442bc45740 1 --2- 172.21.15.19:0/1539728641 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x55bd9221efb0 0x55bd92220420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.633+0000 7f442bc45740 4 client.15297 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T12:07:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.633+0000 7f442bc45740 1 -- 172.21.15.19:0/1539728641 --> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd92176670 con 0x55bd9221efb0 2024-04-25T12:07:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.633+0000 7f442259c640 1 --2- 172.21.15.19:0/1539728641 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x55bd9221efb0 0x55bd92220420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.633+0000 7f442259c640 1 --2- 172.21.15.19:0/1539728641 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x55bd9221efb0 0x55bd92220420 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dcf328bccc25f0c3 server_cookie=968cf494a5e9580b in_seq=0 out_seq=0 2024-04-25T12:07:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.633+0000 7f4409ffb640 10 client.15297 ms_handle_connect on v2:172.21.15.169:6834/3316630328 2024-04-25T12:07:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.634+0000 7f4409ffb640 1 -- 172.21.15.19:0/1539728641 <== mds.3 v2:172.21.15.169:6834/3316630328 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd92176670 con 0x55bd9221efb0 2024-04-25T12:07:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.634+0000 7f4409ffb640 10 client.15297 handle_command_reply: tid=0 2024-04-25T12:07:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.634+0000 7f442bc45740 1 client.15297 shutdown 2024-04-25T12:07:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.635+0000 7f442bc45740 1 -- 172.21.15.19:0/1539728641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209160 msgr2=0x55bd92212920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.635+0000 7f442bc45740 1 --2- 172.21.15.19:0/1539728641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209160 0x55bd92212920 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4418004700 tx=0x7f4418004250 comp rx=0 tx=0).stop 2024-04-25T12:07:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.635+0000 7f442bc45740 1 -- 172.21.15.19:0/1539728641 shutdown_connections 2024-04-25T12:07:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.635+0000 7f442bc45740 1 --2- 172.21.15.19:0/1539728641 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x55bd9221efb0 0x55bd92220420 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.635+0000 7f442bc45740 1 --2- 172.21.15.19:0/1539728641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92209b20 0x55bd92219960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.635+0000 7f442bc45740 1 --2- 172.21.15.19:0/1539728641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209160 0x55bd92212920 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.635+0000 7f442bc45740 1 --2- 172.21.15.19:0/1539728641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92207f90 0x55bd922123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.635+0000 7f4409ffb640 0 client.15297 ms_handle_reset on v2:172.21.15.169:6834/3316630328 2024-04-25T12:07:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.635+0000 7f442bc45740 1 -- 172.21.15.19:0/1539728641 >> 172.21.15.19:0/1539728641 conn(0x55bd921ff070 msgr2=0x55bd921fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.635+0000 7f442bc45740 1 -- 172.21.15.19:0/1539728641 shutdown_connections 2024-04-25T12:07:00.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.635+0000 7f442bc45740 1 -- 172.21.15.19:0/1539728641 wait complete. 2024-04-25T12:07:00.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.635+0000 7f442bc45740 20 client.15297 trim_cache size 0 max 16384 2024-04-25T12:07:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f442bc45740 1 Processor -- start 2024-04-25T12:07:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f442bc45740 1 -- start start 2024-04-25T12:07:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f442bc45740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92207f90 0x55bd92210220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f442bc45740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92209160 0x55bd92219920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f442bc45740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209b20 0x55bd92211ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f442bc45740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bd9221e760 con 0x55bd92209b20 2024-04-25T12:07:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f442bc45740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bd9221ee30 con 0x55bd92207f90 2024-04-25T12:07:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f442bc45740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bd9221f500 con 0x55bd92209160 2024-04-25T12:07:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f4422d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209b20 0x55bd92211ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f4422d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209b20 0x55bd92211ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55478/0 (socket says 172.21.15.19:55478) 2024-04-25T12:07:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f4422d9d640 1 -- 172.21.15.19:0/3455107236 learned_addr learned my addr 172.21.15.19:0/3455107236 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f4422d9d640 1 -- 172.21.15.19:0/3455107236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92209160 msgr2=0x55bd92219920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f4422d9d640 1 --2- 172.21.15.19:0/3455107236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92209160 0x55bd92219920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f4422d9d640 1 -- 172.21.15.19:0/3455107236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92207f90 msgr2=0x55bd92210220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f4422d9d640 1 --2- 172.21.15.19:0/3455107236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92207f90 0x55bd92210220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f4422d9d640 1 -- 172.21.15.19:0/3455107236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd921feea0 con 0x55bd92209b20 2024-04-25T12:07:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.653+0000 7f4422d9d640 1 --2- 172.21.15.19:0/3455107236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209b20 0x55bd92211ff0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f441c035000 tx=0x7f441c041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9793ff0c8c4d863 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.654+0000 7f440bfff640 1 -- 172.21.15.19:0/3455107236 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c04f070 con 0x55bd92209b20 2024-04-25T12:07:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.654+0000 7f440bfff640 1 -- 172.21.15.19:0/3455107236 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f441c036040 con 0x55bd92209b20 2024-04-25T12:07:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.654+0000 7f440bfff640 1 -- 172.21.15.19:0/3455107236 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c02f9e0 con 0x55bd92209b20 2024-04-25T12:07:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.654+0000 7f442bc45740 1 -- 172.21.15.19:0/3455107236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209b20 msgr2=0x55bd92211ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.654+0000 7f442bc45740 1 --2- 172.21.15.19:0/3455107236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209b20 0x55bd92211ff0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f441c035000 tx=0x7f441c041040 comp rx=0 tx=0).stop 2024-04-25T12:07:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.654+0000 7f442bc45740 1 -- 172.21.15.19:0/3455107236 shutdown_connections 2024-04-25T12:07:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.654+0000 7f442bc45740 1 --2- 172.21.15.19:0/3455107236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92209b20 0x55bd92211ff0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.654+0000 7f442bc45740 1 --2- 172.21.15.19:0/3455107236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92209160 0x55bd92219920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.654+0000 7f442bc45740 1 --2- 172.21.15.19:0/3455107236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92207f90 0x55bd92210220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.654+0000 7f442bc45740 1 -- 172.21.15.19:0/3455107236 >> 172.21.15.19:0/3455107236 conn(0x55bd921ff070 msgr2=0x55bd921ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.654+0000 7f442bc45740 1 -- 172.21.15.19:0/3455107236 shutdown_connections 2024-04-25T12:07:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.654+0000 7f442bc45740 1 -- 172.21.15.19:0/3455107236 wait complete. 2024-04-25T12:07:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.654+0000 7f442bc45740 1 -- start start 2024-04-25T12:07:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442bc45740 1 Processor -- start 2024-04-25T12:07:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442bc45740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92207f90 0x55bd92214b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442bc45740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92209160 0x55bd922150c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442bc45740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92209b20 0x55bd92218040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442bc45740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bd922202a0 con 0x55bd92207f90 2024-04-25T12:07:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442bc45740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bd9221fbd0 con 0x55bd92209160 2024-04-25T12:07:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442bc45740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x55bd92220970 con 0x55bd92209b20 2024-04-25T12:07:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442bc45740 10 client.0 _ll_register_callbacks cb 0x7f441c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:07:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92207f90 0x55bd92214b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92207f90 0x55bd92214b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55490/0 (socket says 172.21.15.19:55490) 2024-04-25T12:07:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442259c640 1 -- 172.21.15.19:0/3742073527 learned_addr learned my addr 172.21.15.19:0/3742073527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f4422d9d640 1 --2- 172.21.15.19:0/3742073527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92209b20 0x55bd92218040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f4421d9b640 1 --2- 172.21.15.19:0/3742073527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92209160 0x55bd922150c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442259c640 1 -- 172.21.15.19:0/3742073527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92209b20 msgr2=0x55bd92218040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442259c640 1 --2- 172.21.15.19:0/3742073527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92209b20 0x55bd92218040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442259c640 1 -- 172.21.15.19:0/3742073527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92209160 msgr2=0x55bd922150c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442259c640 1 --2- 172.21.15.19:0/3742073527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92209160 0x55bd922150c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442259c640 1 -- 172.21.15.19:0/3742073527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd92218630 con 0x55bd92207f90 2024-04-25T12:07:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f4422d9d640 1 --2- 172.21.15.19:0/3742073527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92209b20 0x55bd92218040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:07:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442259c640 1 --2- 172.21.15.19:0/3742073527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92207f90 0x55bd92214b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f441409c2d0 tx=0x7f441400dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9a1185b181b53c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f4409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:07:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f4409ffb640 1 -- 172.21.15.19:0/3742073527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414020040 con 0x55bd92207f90 2024-04-25T12:07:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f4409ffb640 1 -- 172.21.15.19:0/3742073527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f44140989b0 con 0x55bd92207f90 2024-04-25T12:07:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.655+0000 7f442bc45740 1 -- 172.21.15.19:0/3742073527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f441c036040 con 0x55bd92207f90 2024-04-25T12:07:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.656+0000 7f4409ffb640 1 -- 172.21.15.19:0/3742073527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f44140a4fb0 con 0x55bd92207f90 2024-04-25T12:07:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.656+0000 7f442bc45740 10 client.15303 fetch_fsmap learned FSMap version 26 2024-04-25T12:07:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.656+0000 7f442bc45740 1 -- 172.21.15.19:0/3742073527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd922199a0 con 0x55bd92207f90 2024-04-25T12:07:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.656+0000 7f4409ffb640 1 -- 172.21.15.19:0/3742073527 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f441c036040 con 0x55bd92207f90 2024-04-25T12:07:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.656+0000 7f4409ffb640 1 -- 172.21.15.19:0/3742073527 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4414098090 con 0x55bd92207f90 2024-04-25T12:07:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.656+0000 7f442bc45740 10 client.15303 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:07:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.656+0000 7f442bc45740 10 client.15303 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T12:07:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.656+0000 7f442bc45740 20 client.15303 populate_metadata read hostname 'smithi019' 2024-04-25T12:07:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.656+0000 7f442bc45740 1 --2- 172.21.15.19:0/3742073527 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x55bd92289650 0x55bd92288360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.656+0000 7f442bc45740 4 client.15303 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T12:07:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.656+0000 7f442bc45740 1 -- 172.21.15.19:0/3742073527 --> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] -- command(tid 0: {"prefix": "status"}) -- 0x55bd921704e0 con 0x55bd92289650 2024-04-25T12:07:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.656+0000 7f4421d9b640 1 --2- 172.21.15.19:0/3742073527 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x55bd92289650 0x55bd92288360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f4409ffb640 10 client.15303 ms_handle_connect on v2:172.21.15.169:6834/3316630328 2024-04-25T12:07:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f4421d9b640 1 --2- 172.21.15.19:0/3742073527 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x55bd92289650 0x55bd92288360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b0239fe2a51fe0d6 server_cookie=59d314abfa996074 in_seq=0 out_seq=0 2024-04-25T12:07:00.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f4409ffb640 1 -- 172.21.15.19:0/3742073527 <== mds.3 v2:172.21.15.169:6834/3316630328 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bd921704e0 con 0x55bd92289650 2024-04-25T12:07:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f4409ffb640 10 client.15303 handle_command_reply: tid=0 2024-04-25T12:07:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f442bc45740 1 client.15303 shutdown 2024-04-25T12:07:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f442bc45740 1 -- 172.21.15.19:0/3742073527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92207f90 msgr2=0x55bd92214b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f442bc45740 1 --2- 172.21.15.19:0/3742073527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92207f90 0x55bd92214b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f441409c2d0 tx=0x7f441400dfa0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f442bc45740 1 -- 172.21.15.19:0/3742073527 shutdown_connections 2024-04-25T12:07:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f442bc45740 1 --2- 172.21.15.19:0/3742073527 >> [v2:172.21.15.169:6834/3316630328,v1:172.21.15.169:6835/3316630328] conn(0x55bd92289650 0x55bd92288360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f442bc45740 1 --2- 172.21.15.19:0/3742073527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55bd92209b20 0x55bd92218040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f442bc45740 1 --2- 172.21.15.19:0/3742073527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd92209160 0x55bd922150c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f442bc45740 1 --2- 172.21.15.19:0/3742073527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bd92207f90 0x55bd92214b80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f4409ffb640 0 client.15303 ms_handle_reset on v2:172.21.15.169:6834/3316630328 2024-04-25T12:07:00.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f442bc45740 1 -- 172.21.15.19:0/3742073527 >> 172.21.15.19:0/3742073527 conn(0x55bd921ff070 msgr2=0x55bd9220cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f442bc45740 1 -- 172.21.15.19:0/3742073527 shutdown_connections 2024-04-25T12:07:00.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f442bc45740 1 -- 172.21.15.19:0/3742073527 wait complete. 2024-04-25T12:07:00.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.657+0000 7f442bc45740 20 client.15303 trim_cache size 0 max 16384 2024-04-25T12:07:00.663 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:07:00.663 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "be62e290-02fa-11ef-bc93-c7b262605968", 2024-04-25T12:07:00.663 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-04-25T12:07:00.663 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24484, 2024-04-25T12:07:00.664 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T12:07:00.664 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T12:07:00.664 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T12:07:00.664 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 110.94530645899999, 2024-04-25T12:07:00.664 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-25T12:07:00.664 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 87, 2024-04-25T12:07:00.664 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T12:07:00.664 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 140.105110577 2024-04-25T12:07:00.664 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:07:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.658+0000 7f4428f36640 1 -- 172.21.15.19:0/1857023117 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f440c0a3c30 msgr2=0x7f440c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.658+0000 7f4428f36640 1 --2- 172.21.15.19:0/1857023117 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f440c0a3c30 0x7f440c0a60f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f441c031a70 tx=0x7f441c0095c0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.658+0000 7f4428f36640 1 -- 172.21.15.19:0/1857023117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424100e90 msgr2=0x7f4424198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.658+0000 7f4428f36640 1 --2- 172.21.15.19:0/1857023117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424100e90 0x7f4424198190 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f441400d950 tx=0x7f441400de10 comp rx=0 tx=0).stop 2024-04-25T12:07:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.658+0000 7f4428f36640 1 -- 172.21.15.19:0/1857023117 shutdown_connections 2024-04-25T12:07:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.658+0000 7f4428f36640 1 --2- 172.21.15.19:0/1857023117 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f440c0a3c30 0x7f440c0a60f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.658+0000 7f4428f36640 1 --2- 172.21.15.19:0/1857023117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44241029a0 0x7f442419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.658+0000 7f4428f36640 1 --2- 172.21.15.19:0/1857023117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4424101fe0 0x7f44241986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.658+0000 7f4428f36640 1 --2- 172.21.15.19:0/1857023117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4424100e90 0x7f4424198190 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.658+0000 7f4428f36640 1 -- 172.21.15.19:0/1857023117 >> 172.21.15.19:0/1857023117 conn(0x7f44240fc400 msgr2=0x7f44240fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.658+0000 7f4428f36640 1 -- 172.21.15.19:0/1857023117 shutdown_connections 2024-04-25T12:07:00.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.658+0000 7f4428f36640 1 -- 172.21.15.19:0/1857023117 wait complete. 2024-04-25T12:07:00.667 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T12:07:00.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.774+0000 7fb6b4176640 1 -- 172.21.15.19:0/920134270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6ac075f10 msgr2=0x7fb6ac074360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.774+0000 7fb6b4176640 1 --2- 172.21.15.19:0/920134270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6ac075f10 0x7fb6ac074360 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009900 tx=0x7fb6a802a480 comp rx=0 tx=0).stop 2024-04-25T12:07:00.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.774+0000 7fb6b4176640 1 -- 172.21.15.19:0/920134270 shutdown_connections 2024-04-25T12:07:00.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.774+0000 7fb6b4176640 1 --2- 172.21.15.19:0/920134270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ac109e60 0x7fb6ac10c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.774+0000 7fb6b4176640 1 --2- 172.21.15.19:0/920134270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6ac074930 0x7fb6ac074d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.774+0000 7fb6b4176640 1 --2- 172.21.15.19:0/920134270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6ac075f10 0x7fb6ac074360 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.774+0000 7fb6b4176640 1 -- 172.21.15.19:0/920134270 >> 172.21.15.19:0/920134270 conn(0x7fb6ac0fd520 msgr2=0x7fb6ac0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.774+0000 7fb6b4176640 1 -- 172.21.15.19:0/920134270 shutdown_connections 2024-04-25T12:07:00.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.774+0000 7fb6b4176640 1 -- 172.21.15.19:0/920134270 wait complete. 2024-04-25T12:07:00.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b4176640 1 Processor -- start 2024-04-25T12:07:00.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b4176640 1 -- start start 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b4176640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ac074930 0x7fb6ac19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b4176640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6ac075f10 0x7fb6ac19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b4176640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6ac109e60 0x7fb6ac1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b4176640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac10fe00 con 0x7fb6ac109e60 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b4176640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac10fc80 con 0x7fb6ac074930 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b4176640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac10ff80 con 0x7fb6ac075f10 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b2713640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6ac109e60 0x7fb6ac1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b2713640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6ac109e60 0x7fb6ac1a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55508/0 (socket says 172.21.15.19:55508) 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b2713640 1 -- 172.21.15.19:0/599632702 learned_addr learned my addr 172.21.15.19:0/599632702 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b1f12640 1 --2- 172.21.15.19:0/599632702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ac074930 0x7fb6ac19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b1711640 1 --2- 172.21.15.19:0/599632702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6ac075f10 0x7fb6ac19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b2713640 1 -- 172.21.15.19:0/599632702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6ac075f10 msgr2=0x7fb6ac19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b2713640 1 --2- 172.21.15.19:0/599632702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6ac075f10 0x7fb6ac19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b2713640 1 -- 172.21.15.19:0/599632702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ac074930 msgr2=0x7fb6ac19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b2713640 1 --2- 172.21.15.19:0/599632702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ac074930 0x7fb6ac19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.775+0000 7fb6b2713640 1 -- 172.21.15.19:0/599632702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ac1a20c0 con 0x7fb6ac109e60 2024-04-25T12:07:00.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.776+0000 7fb6b2713640 1 --2- 172.21.15.19:0/599632702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6ac109e60 0x7fb6ac1a1a50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400eb10 tx=0x7fb6a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.776+0000 7fb6a2ffd640 1 -- 172.21.15.19:0/599632702 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4004b20 con 0x7fb6ac109e60 2024-04-25T12:07:00.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.776+0000 7fb6b4176640 1 -- 172.21.15.19:0/599632702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ac1a23b0 con 0x7fb6ac109e60 2024-04-25T12:07:00.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.776+0000 7fb6a2ffd640 1 -- 172.21.15.19:0/599632702 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb6a4004cc0 con 0x7fb6ac109e60 2024-04-25T12:07:00.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.776+0000 7fb6a2ffd640 1 -- 172.21.15.19:0/599632702 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4010720 con 0x7fb6ac109e60 2024-04-25T12:07:00.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.776+0000 7fb6b4176640 1 -- 172.21.15.19:0/599632702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ac1a2900 con 0x7fb6ac109e60 2024-04-25T12:07:00.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.779+0000 7fb6a2ffd640 1 -- 172.21.15.19:0/599632702 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb6a4005020 con 0x7fb6ac109e60 2024-04-25T12:07:00.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.779+0000 7fb6b6f69740 1 Processor -- start 2024-04-25T12:07:00.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.779+0000 7fb6b6f69740 1 -- start start 2024-04-25T12:07:00.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.779+0000 7fb6b6f69740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786ab80 0x5651a786afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.779+0000 7fb6b6f69740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786bd50 0x5651a786c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.779+0000 7fb6b6f69740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 0x5651a7872fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.779+0000 7fb6b6f69740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5651a77cfc50 con 0x5651a786bd50 2024-04-25T12:07:00.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.779+0000 7fb6b6f69740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5651a776c830 con 0x5651a786c710 2024-04-25T12:07:00.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.779+0000 7fb6b6f69740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5651a77cbc10 con 0x5651a786ab80 2024-04-25T12:07:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.779+0000 7fb6b1f12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786bd50 0x5651a786c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.779+0000 7fb6b1f12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786bd50 0x5651a786c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55516/0 (socket says 172.21.15.19:55516) 2024-04-25T12:07:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.779+0000 7fb6b1f12640 1 -- 172.21.15.19:0/1466604687 learned_addr learned my addr 172.21.15.19:0/1466604687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.779+0000 7fb6b1711640 1 --2- 172.21.15.19:0/1466604687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786ab80 0x5651a786afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.779+0000 7fb6b1711640 1 --2- 172.21.15.19:0/1466604687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 0x5651a7872fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.780+0000 7fb6a2ffd640 1 --2- 172.21.15.19:0/599632702 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb6980a3bb0 0x7fb6980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.780+0000 7fb6b1f12640 1 -- 172.21.15.19:0/1466604687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786ab80 msgr2=0x5651a786afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.780+0000 7fb6b1f12640 1 --2- 172.21.15.19:0/1466604687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786ab80 0x5651a786afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.780+0000 7fb6b1f12640 1 -- 172.21.15.19:0/1466604687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 msgr2=0x5651a7872fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.780+0000 7fb6b1f12640 1 --2- 172.21.15.19:0/1466604687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 0x5651a7872fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.780+0000 7fb6b1f12640 1 -- 172.21.15.19:0/1466604687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a74e22d0 con 0x5651a786bd50 2024-04-25T12:07:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.780+0000 7fb6b1711640 1 --2- 172.21.15.19:0/1466604687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 0x5651a7872fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:07:00.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.780+0000 7fb6b2713640 1 --2- 172.21.15.19:0/599632702 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb6980a3bb0 0x7fb6980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.780+0000 7fb6b1f12640 1 --2- 172.21.15.19:0/1466604687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786bd50 0x5651a786c1d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009900 tx=0x7fb6a8031b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202734cd5b5ace89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.780+0000 7fb6a2ffd640 1 -- 172.21.15.19:0/599632702 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fb6a40d09b0 con 0x7fb6ac109e60 2024-04-25T12:07:00.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.780+0000 7fb697fff640 1 -- 172.21.15.19:0/1466604687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8040070 con 0x5651a786bd50 2024-04-25T12:07:00.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.780+0000 7fb697fff640 1 -- 172.21.15.19:0/1466604687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb6a8004030 con 0x5651a786bd50 2024-04-25T12:07:00.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.780+0000 7fb697fff640 1 -- 172.21.15.19:0/1466604687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8035040 con 0x5651a786bd50 2024-04-25T12:07:00.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.781+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1466604687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786bd50 msgr2=0x5651a786c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.781+0000 7fb6b2713640 1 --2- 172.21.15.19:0/599632702 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb6980a3bb0 0x7fb6980a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6a409cc00 tx=0x7fb6a406f7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.781+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1466604687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786bd50 0x5651a786c1d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009900 tx=0x7fb6a8031b40 comp rx=0 tx=0).stop 2024-04-25T12:07:00.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.781+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1466604687 shutdown_connections 2024-04-25T12:07:00.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.781+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1466604687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 0x5651a7872fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.781+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1466604687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786bd50 0x5651a786c1d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.781+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1466604687 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786ab80 0x5651a786afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.781+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1466604687 >> 172.21.15.19:0/1466604687 conn(0x5651a7866070 msgr2=0x5651a7866470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.781+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1466604687 shutdown_connections 2024-04-25T12:07:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.781+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1466604687 wait complete. 2024-04-25T12:07:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.781+0000 7fb6b6f69740 1 -- start start 2024-04-25T12:07:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.781+0000 7fb6b6f69740 1 Processor -- start 2024-04-25T12:07:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b6f69740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786ab80 0x5651a7874f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b6f69740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786bd50 0x5651a78754c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b6f69740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 0x5651a787c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b6f69740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5651a78736c0 con 0x5651a786bd50 2024-04-25T12:07:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b6f69740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5651a776a470 con 0x5651a786c710 2024-04-25T12:07:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b6f69740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5651a7873840 con 0x5651a786ab80 2024-04-25T12:07:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b6f69740 10 client.0 _ll_register_callbacks cb 0x5651a778ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:07:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b1f12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786bd50 0x5651a78754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b1f12640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786bd50 0x5651a78754c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55532/0 (socket says 172.21.15.19:55532) 2024-04-25T12:07:00.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b1f12640 1 -- 172.21.15.19:0/1640985903 learned_addr learned my addr 172.21.15.19:0/1640985903 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b1711640 1 --2- 172.21.15.19:0/1640985903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786ab80 0x5651a7874f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b1711640 1 --2- 172.21.15.19:0/1640985903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 0x5651a787c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b1f12640 1 -- 172.21.15.19:0/1640985903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786ab80 msgr2=0x5651a7874f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b1f12640 1 --2- 172.21.15.19:0/1640985903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786ab80 0x5651a7874f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b1f12640 1 -- 172.21.15.19:0/1640985903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 msgr2=0x5651a787c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b1f12640 1 --2- 172.21.15.19:0/1640985903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 0x5651a787c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b1f12640 1 -- 172.21.15.19:0/1640985903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a75ac6b0 con 0x5651a786bd50 2024-04-25T12:07:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b1711640 1 --2- 172.21.15.19:0/1640985903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786ab80 0x5651a7874f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:07:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b1f12640 1 --2- 172.21.15.19:0/1640985903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786bd50 0x5651a78754c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8005c40 tx=0x7fb6a8005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca3c05c52c5b1bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6957fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:07:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6957fa640 1 -- 172.21.15.19:0/1640985903 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a804c020 con 0x5651a786bd50 2024-04-25T12:07:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6957fa640 1 -- 172.21.15.19:0/1640985903 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fb6a8031810 con 0x5651a786bd50 2024-04-25T12:07:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.782+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1640985903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6a8004030 con 0x5651a786bd50 2024-04-25T12:07:00.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.783+0000 7fb6957fa640 1 -- 172.21.15.19:0/1640985903 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb6a8057050 con 0x5651a786bd50 2024-04-25T12:07:00.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.783+0000 7fb6b6f69740 10 client.15315 fetch_fsmap learned FSMap version 26 2024-04-25T12:07:00.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.783+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1640985903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5651a787f870 con 0x5651a786bd50 2024-04-25T12:07:00.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.783+0000 7fb6957fa640 1 -- 172.21.15.19:0/1640985903 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb6a8004030 con 0x5651a786bd50 2024-04-25T12:07:00.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.783+0000 7fb6957fa640 1 -- 172.21.15.19:0/1640985903 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb6a8003890 con 0x5651a786bd50 2024-04-25T12:07:00.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.783+0000 7fb6b6f69740 10 client.15315 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:07:00.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.783+0000 7fb6b6f69740 10 client.15315 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T12:07:00.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.783+0000 7fb6b6f69740 20 client.15315 populate_metadata read hostname 'smithi019' 2024-04-25T12:07:00.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.783+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1640985903 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5651a7881b00 0x5651a7882f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.783+0000 7fb6b6f69740 4 client.15315 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T12:07:00.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.783+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1640985903 --> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651a77dd670 con 0x5651a7881b00 2024-04-25T12:07:00.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.783+0000 7fb6b1711640 1 --2- 172.21.15.19:0/1640985903 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5651a7881b00 0x5651a7882f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.784+0000 7fb6b1711640 1 --2- 172.21.15.19:0/1640985903 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5651a7881b00 0x5651a7882f70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d566584d0ec858a4 server_cookie=f556f8d86c7e85e0 in_seq=0 out_seq=0 2024-04-25T12:07:00.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.784+0000 7fb6957fa640 10 client.15315 ms_handle_connect on v2:172.21.15.169:6836/1253767842 2024-04-25T12:07:00.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.784+0000 7fb6957fa640 1 -- 172.21.15.19:0/1640985903 <== mds.2 v2:172.21.15.169:6836/1253767842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651a77dd670 con 0x5651a7881b00 2024-04-25T12:07:00.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.784+0000 7fb6957fa640 10 client.15315 handle_command_reply: tid=0 2024-04-25T12:07:00.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.785+0000 7fb6b6f69740 1 client.15315 shutdown 2024-04-25T12:07:00.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.785+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1640985903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786bd50 msgr2=0x5651a78754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.785+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1640985903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786bd50 0x5651a78754c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8005c40 tx=0x7fb6a8005750 comp rx=0 tx=0).stop 2024-04-25T12:07:00.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.785+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1640985903 shutdown_connections 2024-04-25T12:07:00.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.785+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1640985903 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5651a7881b00 0x5651a7882f70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.785+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1640985903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 0x5651a787c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.785+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1640985903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786bd50 0x5651a78754c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.785+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1640985903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786ab80 0x5651a7874f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.785+0000 7fb6957fa640 0 client.15315 ms_handle_reset on v2:172.21.15.169:6836/1253767842 2024-04-25T12:07:00.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.785+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1640985903 >> 172.21.15.19:0/1640985903 conn(0x5651a7866070 msgr2=0x5651a78651d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.785+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1640985903 shutdown_connections 2024-04-25T12:07:00.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.785+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1640985903 wait complete. 2024-04-25T12:07:00.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.785+0000 7fb6b6f69740 20 client.15315 trim_cache size 0 max 16384 2024-04-25T12:07:00.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b6f69740 1 Processor -- start 2024-04-25T12:07:00.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b6f69740 1 -- start start 2024-04-25T12:07:00.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b6f69740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786ab80 0x5651a78610b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b6f69740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786bd50 0x5651a787c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b6f69740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786c710 0x5651a7873be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b6f69740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5651a78812b0 con 0x5651a786ab80 2024-04-25T12:07:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b6f69740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5651a7881980 con 0x5651a786bd50 2024-04-25T12:07:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b6f69740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5651a7882050 con 0x5651a786c710 2024-04-25T12:07:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b1711640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786ab80 0x5651a78610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b1711640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786ab80 0x5651a78610b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55546/0 (socket says 172.21.15.19:55546) 2024-04-25T12:07:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b1711640 1 -- 172.21.15.19:0/3001686774 learned_addr learned my addr 172.21.15.19:0/3001686774 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b1f12640 1 --2- 172.21.15.19:0/3001686774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786bd50 0x5651a787c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b1711640 1 -- 172.21.15.19:0/3001686774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786c710 msgr2=0x5651a7873be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b1711640 1 --2- 172.21.15.19:0/3001686774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786c710 0x5651a7873be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b1711640 1 -- 172.21.15.19:0/3001686774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786bd50 msgr2=0x5651a787c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b1711640 1 --2- 172.21.15.19:0/3001686774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786bd50 0x5651a787c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b1711640 1 -- 172.21.15.19:0/3001686774 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a7876100 con 0x5651a786ab80 2024-04-25T12:07:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.803+0000 7fb6b1f12640 1 --2- 172.21.15.19:0/3001686774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786bd50 0x5651a787c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:07:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.804+0000 7fb6b1711640 1 --2- 172.21.15.19:0/3001686774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786ab80 0x5651a78610b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb69c005fd0 tx=0x7fb69c00c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d0a4e726b58f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.804+0000 7fb6977fe640 1 -- 172.21.15.19:0/3001686774 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c023070 con 0x5651a786ab80 2024-04-25T12:07:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.804+0000 7fb6977fe640 1 -- 172.21.15.19:0/3001686774 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb69c0140d0 con 0x5651a786ab80 2024-04-25T12:07:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.804+0000 7fb6977fe640 1 -- 172.21.15.19:0/3001686774 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c003bf0 con 0x5651a786ab80 2024-04-25T12:07:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.804+0000 7fb6b6f69740 1 -- 172.21.15.19:0/3001686774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786ab80 msgr2=0x5651a78610b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.804+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/3001686774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786ab80 0x5651a78610b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb69c005fd0 tx=0x7fb69c00c4d0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.804+0000 7fb6b6f69740 1 -- 172.21.15.19:0/3001686774 shutdown_connections 2024-04-25T12:07:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.804+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/3001686774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786c710 0x5651a7873be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.804+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/3001686774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786bd50 0x5651a787c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.804+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/3001686774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786ab80 0x5651a78610b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.804+0000 7fb6b6f69740 1 -- 172.21.15.19:0/3001686774 >> 172.21.15.19:0/3001686774 conn(0x5651a7866070 msgr2=0x5651a7866450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.804+0000 7fb6b6f69740 1 -- 172.21.15.19:0/3001686774 shutdown_connections 2024-04-25T12:07:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.804+0000 7fb6b6f69740 1 -- 172.21.15.19:0/3001686774 wait complete. 2024-04-25T12:07:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.805+0000 7fb6b6f69740 1 -- start start 2024-04-25T12:07:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.805+0000 7fb6b6f69740 1 Processor -- start 2024-04-25T12:07:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.805+0000 7fb6b6f69740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786ab80 0x5651a78781e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.805+0000 7fb6b6f69740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786bd50 0x5651a7878720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.805+0000 7fb6b6f69740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 0x5651a787b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.805+0000 7fb6b6f69740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5651a7882df0 con 0x5651a786ab80 2024-04-25T12:07:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.805+0000 7fb6b6f69740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5651a7882720 con 0x5651a786c710 2024-04-25T12:07:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.805+0000 7fb6b6f69740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x5651a78834c0 con 0x5651a786bd50 2024-04-25T12:07:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.805+0000 7fb6b6f69740 10 client.0 _ll_register_callbacks cb 0x7fb69c00ca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:07:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.805+0000 7fb6b1711640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786ab80 0x5651a78781e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.805+0000 7fb6b1711640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786ab80 0x5651a78781e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55556/0 (socket says 172.21.15.19:55556) 2024-04-25T12:07:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.805+0000 7fb6b1711640 1 -- 172.21.15.19:0/1366552376 learned_addr learned my addr 172.21.15.19:0/1366552376 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.805+0000 7fb6b1f12640 1 --2- 172.21.15.19:0/1366552376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786bd50 0x5651a7878720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.805+0000 7fb6b1711640 1 --2- 172.21.15.19:0/1366552376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 0x5651a787b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6b1711640 1 -- 172.21.15.19:0/1366552376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786bd50 msgr2=0x5651a7878720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6b1711640 1 --2- 172.21.15.19:0/1366552376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786bd50 0x5651a7878720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6b1711640 1 -- 172.21.15.19:0/1366552376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 msgr2=0x5651a787b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6b1711640 1 --2- 172.21.15.19:0/1366552376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 0x5651a787b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6b1711640 1 -- 172.21.15.19:0/1366552376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a787bd30 con 0x5651a786ab80 2024-04-25T12:07:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6b1711640 1 --2- 172.21.15.19:0/1366552376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786ab80 0x5651a78781e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00c5f0 tx=0x7fb69c00c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723e707cf99e3890 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6957fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:07:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6957fa640 1 -- 172.21.15.19:0/1366552376 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c031060 con 0x5651a786ab80 2024-04-25T12:07:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6957fa640 1 -- 172.21.15.19:0/1366552376 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fb69c012900 con 0x5651a786ab80 2024-04-25T12:07:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1366552376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb69c0140d0 con 0x5651a786ab80 2024-04-25T12:07:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6957fa640 1 -- 172.21.15.19:0/1366552376 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb69c023040 con 0x5651a786ab80 2024-04-25T12:07:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6b6f69740 10 client.15321 fetch_fsmap learned FSMap version 26 2024-04-25T12:07:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1366552376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5651a78e8010 con 0x5651a786ab80 2024-04-25T12:07:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6957fa640 1 -- 172.21.15.19:0/1366552376 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb69c0140d0 con 0x5651a786ab80 2024-04-25T12:07:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6957fa640 1 -- 172.21.15.19:0/1366552376 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb69c029070 con 0x5651a786ab80 2024-04-25T12:07:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6b6f69740 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:07:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6b6f69740 10 client.15321 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T12:07:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6b6f69740 20 client.15321 populate_metadata read hostname 'smithi019' 2024-04-25T12:07:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.806+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1366552376 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5651a78ec800 0x5651a78eb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.807+0000 7fb6b6f69740 4 client.15321 mds_command: new command op to 24496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.807+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1366552376 --> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] -- command(tid 0: {"prefix": "status"}) -- 0x5651a77d74e0 con 0x5651a78ec800 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.807+0000 7fb6b1f12640 1 --2- 172.21.15.19:0/1366552376 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5651a78ec800 0x5651a78eb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.807+0000 7fb6957fa640 10 client.15321 ms_handle_connect on v2:172.21.15.169:6836/1253767842 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.807+0000 7fb6b1f12640 1 --2- 172.21.15.19:0/1366552376 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5651a78ec800 0x5651a78eb510 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b35f96e7c401031 server_cookie=1f4491f8a084feb4 in_seq=0 out_seq=0 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.807+0000 7fb6957fa640 1 -- 172.21.15.19:0/1366552376 <== mds.2 v2:172.21.15.169:6836/1253767842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5651a77d74e0 con 0x5651a78ec800 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.807+0000 7fb6957fa640 10 client.15321 handle_command_reply: tid=0 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.807+0000 7fb6b6f69740 1 client.15321 shutdown 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.807+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1366552376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786ab80 msgr2=0x5651a78781e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.807+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1366552376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786ab80 0x5651a78781e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00c5f0 tx=0x7fb69c00c540 comp rx=0 tx=0).stop 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1366552376 shutdown_connections 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1366552376 >> [v2:172.21.15.169:6836/1253767842,v1:172.21.15.169:6837/1253767842] conn(0x5651a78ec800 0x5651a78eb510 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1366552376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651a786c710 0x5651a787b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1366552376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651a786bd50 0x5651a7878720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b6f69740 1 --2- 172.21.15.19:0/1366552376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5651a786ab80 0x5651a78781e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6957fa640 0 client.15321 ms_handle_reset on v2:172.21.15.169:6836/1253767842 2024-04-25T12:07:00.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1366552376 >> 172.21.15.19:0/1366552376 conn(0x5651a7866070 msgr2=0x5651a78610b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1366552376 shutdown_connections 2024-04-25T12:07:00.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b6f69740 1 -- 172.21.15.19:0/1366552376 wait complete. 2024-04-25T12:07:00.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b6f69740 20 client.15321 trim_cache size 0 max 16384 2024-04-25T12:07:00.814 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:07:00.814 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "be62e290-02fa-11ef-bc93-c7b262605968", 2024-04-25T12:07:00.814 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-04-25T12:07:00.814 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24496, 2024-04-25T12:07:00.815 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T12:07:00.815 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T12:07:00.815 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T12:07:00.815 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 112.108926627, 2024-04-25T12:07:00.815 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-25T12:07:00.815 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 87, 2024-04-25T12:07:00.815 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T12:07:00.815 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 137.43314838000001 2024-04-25T12:07:00.815 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:07:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b4176640 1 -- 172.21.15.19:0/599632702 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb6980a3bb0 msgr2=0x7fb6980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b4176640 1 --2- 172.21.15.19:0/599632702 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb6980a3bb0 0x7fb6980a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6a409cc00 tx=0x7fb6a406f7b0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b4176640 1 -- 172.21.15.19:0/599632702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6ac109e60 msgr2=0x7fb6ac1a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b4176640 1 --2- 172.21.15.19:0/599632702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6ac109e60 0x7fb6ac1a1a50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400eb10 tx=0x7fb6a400c2d0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b4176640 1 -- 172.21.15.19:0/599632702 shutdown_connections 2024-04-25T12:07:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b4176640 1 --2- 172.21.15.19:0/599632702 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb6980a3bb0 0x7fb6980a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b4176640 1 --2- 172.21.15.19:0/599632702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6ac109e60 0x7fb6ac1a1a50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b4176640 1 --2- 172.21.15.19:0/599632702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6ac075f10 0x7fb6ac19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b4176640 1 --2- 172.21.15.19:0/599632702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6ac074930 0x7fb6ac19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b4176640 1 -- 172.21.15.19:0/599632702 >> 172.21.15.19:0/599632702 conn(0x7fb6ac0fd520 msgr2=0x7fb6ac105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b4176640 1 -- 172.21.15.19:0/599632702 shutdown_connections 2024-04-25T12:07:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.808+0000 7fb6b4176640 1 -- 172.21.15.19:0/599632702 wait complete. 2024-04-25T12:07:00.818 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T12:07:00.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.924+0000 7f3255aa7640 1 -- 172.21.15.19:0/2337331809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32501029a0 msgr2=0x7f32501092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.924+0000 7f3255aa7640 1 --2- 172.21.15.19:0/2337331809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32501029a0 0x7f32501092c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3248009ca0 tx=0x7f324802a8c0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.924+0000 7f3255aa7640 1 -- 172.21.15.19:0/2337331809 shutdown_connections 2024-04-25T12:07:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.924+0000 7f3255aa7640 1 --2- 172.21.15.19:0/2337331809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32501029a0 0x7f32501092c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.924+0000 7f3255aa7640 1 --2- 172.21.15.19:0/2337331809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3250101fe0 0x7f3250102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.924+0000 7f3255aa7640 1 --2- 172.21.15.19:0/2337331809 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3250100e90 0x7f32501012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.924+0000 7f3255aa7640 1 -- 172.21.15.19:0/2337331809 >> 172.21.15.19:0/2337331809 conn(0x7f32500fc400 msgr2=0x7f32500fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.924+0000 7f3255aa7640 1 -- 172.21.15.19:0/2337331809 shutdown_connections 2024-04-25T12:07:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.924+0000 7f3255aa7640 1 -- 172.21.15.19:0/2337331809 wait complete. 2024-04-25T12:07:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.924+0000 7f3255aa7640 1 Processor -- start 2024-04-25T12:07:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f3255aa7640 1 -- start start 2024-04-25T12:07:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f3255aa7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3250100e90 0x7f3250104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f3255aa7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3250101fe0 0x7f3250103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f3255aa7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32501029a0 0x7f3250103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f3255aa7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f325010cb30 con 0x7f3250100e90 2024-04-25T12:07:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f3255aa7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f325010c9b0 con 0x7f32501029a0 2024-04-25T12:07:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f3255aa7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f325010ccb0 con 0x7f3250101fe0 2024-04-25T12:07:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f324f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3250100e90 0x7f3250104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f324f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3250100e90 0x7f3250104e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55576/0 (socket says 172.21.15.19:55576) 2024-04-25T12:07:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f324f7fe640 1 -- 172.21.15.19:0/1728744532 learned_addr learned my addr 172.21.15.19:0/1728744532 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f324effd640 1 --2- 172.21.15.19:0/1728744532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3250101fe0 0x7f3250103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f324ffff640 1 --2- 172.21.15.19:0/1728744532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32501029a0 0x7f3250103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f324f7fe640 1 -- 172.21.15.19:0/1728744532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3250101fe0 msgr2=0x7f3250103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f324f7fe640 1 --2- 172.21.15.19:0/1728744532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3250101fe0 0x7f3250103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f324f7fe640 1 -- 172.21.15.19:0/1728744532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32501029a0 msgr2=0x7f3250103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f324f7fe640 1 --2- 172.21.15.19:0/1728744532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32501029a0 0x7f3250103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.925+0000 7f324f7fe640 1 -- 172.21.15.19:0/1728744532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3250104220 con 0x7f3250100e90 2024-04-25T12:07:00.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.926+0000 7f324f7fe640 1 --2- 172.21.15.19:0/1728744532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3250100e90 0x7f3250104e10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f324000c970 tx=0x7f324000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.926+0000 7f324cff9640 1 -- 172.21.15.19:0/1728744532 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240007bb0 con 0x7f3250100e90 2024-04-25T12:07:00.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.926+0000 7f3255aa7640 1 -- 172.21.15.19:0/1728744532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32501a2390 con 0x7f3250100e90 2024-04-25T12:07:00.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.926+0000 7f324cff9640 1 -- 172.21.15.19:0/1728744532 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3240007d50 con 0x7f3250100e90 2024-04-25T12:07:00.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.926+0000 7f324cff9640 1 -- 172.21.15.19:0/1728744532 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324000f450 con 0x7f3250100e90 2024-04-25T12:07:00.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.926+0000 7f3255aa7640 1 -- 172.21.15.19:0/1728744532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32501a2890 con 0x7f3250100e90 2024-04-25T12:07:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f324cff9640 1 -- 172.21.15.19:0/1728744532 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f32400046c0 con 0x7f3250100e90 2024-04-25T12:07:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f32587c4740 1 Processor -- start 2024-04-25T12:07:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f32587c4740 1 -- start start 2024-04-25T12:07:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f32587c4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05ab80 0x563dff05afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f32587c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff05c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f32587c4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 0x563dff062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f32587c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563dfefbfc50 con 0x563dff05bd50 2024-04-25T12:07:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f32587c4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563dfef5c830 con 0x563dff05ab80 2024-04-25T12:07:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f32587c4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563dfefbbc10 con 0x563dff05c710 2024-04-25T12:07:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f324f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff05c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f324f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff05c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55582/0 (socket says 172.21.15.19:55582) 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f324f7fe640 1 -- 172.21.15.19:0/2748566937 learned_addr learned my addr 172.21.15.19:0/2748566937 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f324ffff640 1 --2- 172.21.15.19:0/2748566937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05ab80 0x563dff05afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f324effd640 1 --2- 172.21.15.19:0/2748566937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 0x563dff062fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f324cff9640 1 --2- 172.21.15.19:0/1728744532 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f32380a3bb0 0x7f32380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f324ffff640 1 --2- 172.21.15.19:0/1728744532 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f32380a3bb0 0x7f32380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f324f7fe640 1 -- 172.21.15.19:0/2748566937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 msgr2=0x563dff062fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f324f7fe640 1 --2- 172.21.15.19:0/2748566937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 0x563dff062fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f324f7fe640 1 -- 172.21.15.19:0/2748566937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05ab80 msgr2=0x563dff05afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f324f7fe640 1 --2- 172.21.15.19:0/2748566937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05ab80 0x563dff05afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f324cff9640 1 -- 172.21.15.19:0/1728744532 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f32400d01d0 con 0x7f3250100e90 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.929+0000 7f324f7fe640 1 -- 172.21.15.19:0/2748566937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dfecd22d0 con 0x563dff05bd50 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.930+0000 7f324f7fe640 1 --2- 172.21.15.19:0/2748566937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff05c1d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f32400d3fd0 tx=0x7f32400049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4e1d405d8cd3ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.930+0000 7f324ffff640 1 --2- 172.21.15.19:0/1728744532 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f32380a3bb0 0x7f32380a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3248009430 tx=0x7f324802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.930+0000 7f3235ffb640 1 -- 172.21.15.19:0/2748566937 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324001d070 con 0x563dff05bd50 2024-04-25T12:07:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.930+0000 7f3235ffb640 1 -- 172.21.15.19:0/2748566937 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f32400056a0 con 0x563dff05bd50 2024-04-25T12:07:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.930+0000 7f3235ffb640 1 -- 172.21.15.19:0/2748566937 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32400a0840 con 0x563dff05bd50 2024-04-25T12:07:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.930+0000 7f32587c4740 1 -- 172.21.15.19:0/2748566937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 msgr2=0x563dff05c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.930+0000 7f32587c4740 1 --2- 172.21.15.19:0/2748566937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff05c1d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f32400d3fd0 tx=0x7f32400049f0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 -- 172.21.15.19:0/2748566937 shutdown_connections 2024-04-25T12:07:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 --2- 172.21.15.19:0/2748566937 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 0x563dff062fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 --2- 172.21.15.19:0/2748566937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff05c1d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 --2- 172.21.15.19:0/2748566937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05ab80 0x563dff05afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 -- 172.21.15.19:0/2748566937 >> 172.21.15.19:0/2748566937 conn(0x563dff056070 msgr2=0x563dff056470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 -- 172.21.15.19:0/2748566937 shutdown_connections 2024-04-25T12:07:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 -- 172.21.15.19:0/2748566937 wait complete. 2024-04-25T12:07:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 -- start start 2024-04-25T12:07:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 Processor -- start 2024-04-25T12:07:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05ab80 0x563dff064fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff065510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 0x563dff06c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563dff0636c0 con 0x563dff05bd50 2024-04-25T12:07:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563dfef5a470 con 0x563dff05ab80 2024-04-25T12:07:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563dff063840 con 0x563dff05c710 2024-04-25T12:07:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f32587c4740 10 client.0 _ll_register_callbacks cb 0x563dfef7ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:07:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f324effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff065510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f324effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff065510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55584/0 (socket says 172.21.15.19:55584) 2024-04-25T12:07:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.931+0000 7f324effd640 1 -- 172.21.15.19:0/3066662668 learned_addr learned my addr 172.21.15.19:0/3066662668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f324f7fe640 1 --2- 172.21.15.19:0/3066662668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05ab80 0x563dff064fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f324effd640 1 -- 172.21.15.19:0/3066662668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 msgr2=0x563dff06c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f324effd640 1 --2- 172.21.15.19:0/3066662668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 0x563dff06c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f324effd640 1 -- 172.21.15.19:0/3066662668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05ab80 msgr2=0x563dff064fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f324effd640 1 --2- 172.21.15.19:0/3066662668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05ab80 0x563dff064fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f324effd640 1 -- 172.21.15.19:0/3066662668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dfed9c6b0 con 0x563dff05bd50 2024-04-25T12:07:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f324effd640 1 --2- 172.21.15.19:0/3066662668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff065510 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f324400ba00 tx=0x7f324400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47a5945470f24e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f322effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:07:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f322effd640 1 -- 172.21.15.19:0/3066662668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244003dc0 con 0x563dff05bd50 2024-04-25T12:07:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f322effd640 1 -- 172.21.15.19:0/3066662668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f3244005020 con 0x563dff05bd50 2024-04-25T12:07:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f32587c4740 1 -- 172.21.15.19:0/3066662668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32400056a0 con 0x563dff05bd50 2024-04-25T12:07:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f322effd640 1 -- 172.21.15.19:0/3066662668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3244005260 con 0x563dff05bd50 2024-04-25T12:07:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f32587c4740 10 client.15333 fetch_fsmap learned FSMap version 26 2024-04-25T12:07:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f32587c4740 1 -- 172.21.15.19:0/3066662668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563dff06f920 con 0x563dff05bd50 2024-04-25T12:07:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.932+0000 7f322effd640 1 -- 172.21.15.19:0/3066662668 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f32400056a0 con 0x563dff05bd50 2024-04-25T12:07:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.933+0000 7f322effd640 1 -- 172.21.15.19:0/3066662668 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f324401b500 con 0x563dff05bd50 2024-04-25T12:07:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.933+0000 7f32587c4740 10 client.15333 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:07:00.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.933+0000 7f32587c4740 10 client.15333 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T12:07:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.933+0000 7f32587c4740 20 client.15333 populate_metadata read hostname 'smithi019' 2024-04-25T12:07:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.933+0000 7f32587c4740 1 --2- 172.21.15.19:0/3066662668 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x563dff072050 0x563dff0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.933+0000 7f32587c4740 4 client.15333 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T12:07:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.933+0000 7f32587c4740 1 -- 172.21.15.19:0/3066662668 --> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dfefcd670 con 0x563dff072050 2024-04-25T12:07:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.933+0000 7f324f7fe640 1 --2- 172.21.15.19:0/3066662668 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x563dff072050 0x563dff0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.933+0000 7f324f7fe640 1 --2- 172.21.15.19:0/3066662668 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x563dff072050 0x563dff0734c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab70c17e419e2577 server_cookie=d80172d82e336a24 in_seq=0 out_seq=0 2024-04-25T12:07:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.933+0000 7f322effd640 10 client.15333 ms_handle_connect on v2:172.21.15.150:6838/2553528303 2024-04-25T12:07:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.934+0000 7f322effd640 1 -- 172.21.15.19:0/3066662668 <== mds.1 v2:172.21.15.150:6838/2553528303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563dfefcd670 con 0x563dff072050 2024-04-25T12:07:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.934+0000 7f322effd640 10 client.15333 handle_command_reply: tid=0 2024-04-25T12:07:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.934+0000 7f32587c4740 1 client.15333 shutdown 2024-04-25T12:07:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.934+0000 7f32587c4740 1 -- 172.21.15.19:0/3066662668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 msgr2=0x563dff065510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.934+0000 7f32587c4740 1 --2- 172.21.15.19:0/3066662668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff065510 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f324400ba00 tx=0x7f324400bec0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.934+0000 7f32587c4740 1 -- 172.21.15.19:0/3066662668 shutdown_connections 2024-04-25T12:07:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.934+0000 7f32587c4740 1 --2- 172.21.15.19:0/3066662668 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x563dff072050 0x563dff0734c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.934+0000 7f32587c4740 1 --2- 172.21.15.19:0/3066662668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 0x563dff06c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.934+0000 7f32587c4740 1 --2- 172.21.15.19:0/3066662668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff065510 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.934+0000 7f322effd640 0 client.15333 ms_handle_reset on v2:172.21.15.150:6838/2553528303 2024-04-25T12:07:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.934+0000 7f32587c4740 1 --2- 172.21.15.19:0/3066662668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05ab80 0x563dff064fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.934+0000 7f32587c4740 1 -- 172.21.15.19:0/3066662668 >> 172.21.15.19:0/3066662668 conn(0x563dff056070 msgr2=0x563dff0551d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.934+0000 7f32587c4740 1 -- 172.21.15.19:0/3066662668 shutdown_connections 2024-04-25T12:07:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.934+0000 7f32587c4740 1 -- 172.21.15.19:0/3066662668 wait complete. 2024-04-25T12:07:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.935+0000 7f32587c4740 20 client.15333 trim_cache size 0 max 16384 2024-04-25T12:07:00.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.953+0000 7f32587c4740 1 Processor -- start 2024-04-25T12:07:00.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.953+0000 7f32587c4740 1 -- start start 2024-04-25T12:07:00.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.953+0000 7f32587c4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05ab80 0x563dff0510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.953+0000 7f32587c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff06c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.953+0000 7f32587c4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 0x563dff064be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.953+0000 7f32587c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563dff071800 con 0x563dff05bd50 2024-04-25T12:07:00.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.953+0000 7f32587c4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563dff071ed0 con 0x563dff05ab80 2024-04-25T12:07:00.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.953+0000 7f32587c4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563dff0725a0 con 0x563dff05c710 2024-04-25T12:07:00.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.953+0000 7f324effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff06c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.953+0000 7f324effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff06c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55596/0 (socket says 172.21.15.19:55596) 2024-04-25T12:07:00.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.953+0000 7f324effd640 1 -- 172.21.15.19:0/4243474121 learned_addr learned my addr 172.21.15.19:0/4243474121 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.954+0000 7f324effd640 1 -- 172.21.15.19:0/4243474121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 msgr2=0x563dff064be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.954+0000 7f324effd640 1 --2- 172.21.15.19:0/4243474121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 0x563dff064be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.954+0000 7f324effd640 1 -- 172.21.15.19:0/4243474121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05ab80 msgr2=0x563dff0510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.954+0000 7f324effd640 1 --2- 172.21.15.19:0/4243474121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05ab80 0x563dff0510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.954+0000 7f324effd640 1 -- 172.21.15.19:0/4243474121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dff05b7d0 con 0x563dff05bd50 2024-04-25T12:07:00.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.954+0000 7f324effd640 1 --2- 172.21.15.19:0/4243474121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff06c4a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3244006010 tx=0x7f3244003a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78dbc62765ef4222 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.954+0000 7f32357fa640 1 -- 172.21.15.19:0/4243474121 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244033020 con 0x563dff05bd50 2024-04-25T12:07:00.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.954+0000 7f32357fa640 1 -- 172.21.15.19:0/4243474121 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3244003e40 con 0x563dff05bd50 2024-04-25T12:07:00.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.954+0000 7f32357fa640 1 -- 172.21.15.19:0/4243474121 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244000e60 con 0x563dff05bd50 2024-04-25T12:07:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.954+0000 7f32587c4740 1 -- 172.21.15.19:0/4243474121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 msgr2=0x563dff06c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.954+0000 7f32587c4740 1 --2- 172.21.15.19:0/4243474121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff06c4a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3244006010 tx=0x7f3244003a40 comp rx=0 tx=0).stop 2024-04-25T12:07:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 -- 172.21.15.19:0/4243474121 shutdown_connections 2024-04-25T12:07:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 --2- 172.21.15.19:0/4243474121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 0x563dff064be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 --2- 172.21.15.19:0/4243474121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05bd50 0x563dff06c4a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 --2- 172.21.15.19:0/4243474121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05ab80 0x563dff0510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 -- 172.21.15.19:0/4243474121 >> 172.21.15.19:0/4243474121 conn(0x563dff056070 msgr2=0x563dff056450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 -- 172.21.15.19:0/4243474121 shutdown_connections 2024-04-25T12:07:00.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 -- 172.21.15.19:0/4243474121 wait complete. 2024-04-25T12:07:00.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 -- start start 2024-04-25T12:07:00.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 Processor -- start 2024-04-25T12:07:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05ab80 0x563dff06a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05bd50 0x563dff0d3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 0x563dff0d6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563dff073340 con 0x563dff05ab80 2024-04-25T12:07:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563dff072c70 con 0x563dff05bd50 2024-04-25T12:07:00.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x563dff073a10 con 0x563dff05c710 2024-04-25T12:07:00.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f32587c4740 10 client.0 _ll_register_callbacks cb 0x7f32440041d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:07:00.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f324f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05ab80 0x563dff06a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f324f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05ab80 0x563dff06a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55612/0 (socket says 172.21.15.19:55612) 2024-04-25T12:07:00.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.955+0000 7f324f7fe640 1 -- 172.21.15.19:0/194155251 learned_addr learned my addr 172.21.15.19:0/194155251 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:00.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f324ffff640 1 --2- 172.21.15.19:0/194155251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 0x563dff0d6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f324effd640 1 --2- 172.21.15.19:0/194155251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05bd50 0x563dff0d3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f324f7fe640 1 -- 172.21.15.19:0/194155251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 msgr2=0x563dff0d6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f324f7fe640 1 --2- 172.21.15.19:0/194155251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 0x563dff0d6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f324f7fe640 1 -- 172.21.15.19:0/194155251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05bd50 msgr2=0x563dff0d3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:00.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f324f7fe640 1 --2- 172.21.15.19:0/194155251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05bd50 0x563dff0d3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f324f7fe640 1 -- 172.21.15.19:0/194155251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dff0d6e30 con 0x563dff05ab80 2024-04-25T12:07:00.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f324effd640 1 --2- 172.21.15.19:0/194155251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05bd50 0x563dff0d3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:07:00.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f324f7fe640 1 --2- 172.21.15.19:0/194155251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05ab80 0x563dff06a3f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f32400988f0 tx=0x7f32400d04f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b75ba32ae515028 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:00.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f322effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:07:00.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f322effd640 1 -- 172.21.15.19:0/194155251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32400cf040 con 0x563dff05ab80 2024-04-25T12:07:00.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f322effd640 1 -- 172.21.15.19:0/194155251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f3240098b00 con 0x563dff05ab80 2024-04-25T12:07:00.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f32587c4740 1 -- 172.21.15.19:0/194155251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3244003e40 con 0x563dff05ab80 2024-04-25T12:07:00.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f322effd640 1 -- 172.21.15.19:0/194155251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f324001d040 con 0x563dff05ab80 2024-04-25T12:07:00.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f32587c4740 10 client.15339 fetch_fsmap learned FSMap version 26 2024-04-25T12:07:00.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f32587c4740 1 -- 172.21.15.19:0/194155251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563dff0d8130 con 0x563dff05ab80 2024-04-25T12:07:00.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.956+0000 7f322effd640 1 -- 172.21.15.19:0/194155251 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f3244003e40 con 0x563dff05ab80 2024-04-25T12:07:00.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.957+0000 7f322effd640 1 -- 172.21.15.19:0/194155251 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f32400981e0 con 0x563dff05ab80 2024-04-25T12:07:00.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.957+0000 7f32587c4740 10 client.15339 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:07:00.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.957+0000 7f32587c4740 10 client.15339 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T12:07:00.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.957+0000 7f32587c4740 20 client.15339 populate_metadata read hostname 'smithi019' 2024-04-25T12:07:00.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.957+0000 7f32587c4740 1 --2- 172.21.15.19:0/194155251 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x563dff0dc810 0x563dff0d9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:00.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.957+0000 7f32587c4740 4 client.15339 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T12:07:00.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.957+0000 7f32587c4740 1 -- 172.21.15.19:0/194155251 --> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] -- command(tid 0: {"prefix": "status"}) -- 0x563dfefc74e0 con 0x563dff0dc810 2024-04-25T12:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.957+0000 7f324effd640 1 --2- 172.21.15.19:0/194155251 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x563dff0dc810 0x563dff0d9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.957+0000 7f322effd640 10 client.15339 ms_handle_connect on v2:172.21.15.150:6838/2553528303 2024-04-25T12:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.957+0000 7f324effd640 1 --2- 172.21.15.19:0/194155251 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x563dff0dc810 0x563dff0d9320 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dec7739cf5d43763 server_cookie=e1f4bba40cc6e351 in_seq=0 out_seq=0 2024-04-25T12:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f322effd640 1 -- 172.21.15.19:0/194155251 <== mds.1 v2:172.21.15.150:6838/2553528303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563dfefc74e0 con 0x563dff0dc810 2024-04-25T12:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f322effd640 10 client.15339 handle_command_reply: tid=0 2024-04-25T12:07:00.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f32587c4740 1 client.15339 shutdown 2024-04-25T12:07:00.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f32587c4740 1 -- 172.21.15.19:0/194155251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05ab80 msgr2=0x563dff06a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f32587c4740 1 --2- 172.21.15.19:0/194155251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05ab80 0x563dff06a3f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f32400988f0 tx=0x7f32400d04f0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f32587c4740 1 -- 172.21.15.19:0/194155251 shutdown_connections 2024-04-25T12:07:00.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f32587c4740 1 --2- 172.21.15.19:0/194155251 >> [v2:172.21.15.150:6838/2553528303,v1:172.21.15.150:6839/2553528303] conn(0x563dff0dc810 0x563dff0d9320 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f32587c4740 1 --2- 172.21.15.19:0/194155251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x563dff05c710 0x563dff0d6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f32587c4740 1 --2- 172.21.15.19:0/194155251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563dff05bd50 0x563dff0d3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f322effd640 0 client.15339 ms_handle_reset on v2:172.21.15.150:6838/2553528303 2024-04-25T12:07:00.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f32587c4740 1 --2- 172.21.15.19:0/194155251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dff05ab80 0x563dff06a3f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f32587c4740 1 -- 172.21.15.19:0/194155251 >> 172.21.15.19:0/194155251 conn(0x563dff056070 msgr2=0x563dff0510b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f32587c4740 1 -- 172.21.15.19:0/194155251 shutdown_connections 2024-04-25T12:07:00.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f32587c4740 1 -- 172.21.15.19:0/194155251 wait complete. 2024-04-25T12:07:00.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f32587c4740 20 client.15339 trim_cache size 0 max 16384 2024-04-25T12:07:00.978 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:07:00.978 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "be62e290-02fa-11ef-bc93-c7b262605968", 2024-04-25T12:07:00.979 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-04-25T12:07:00.979 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24500, 2024-04-25T12:07:00.979 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T12:07:00.979 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T12:07:00.979 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T12:07:00.979 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 113.263226554, 2024-04-25T12:07:00.980 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-25T12:07:00.980 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 87, 2024-04-25T12:07:00.980 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T12:07:00.980 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 123.02811333699999 2024-04-25T12:07:00.980 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:07:00.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f3255aa7640 1 -- 172.21.15.19:0/1728744532 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f32380a3bb0 msgr2=0x7f32380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f3255aa7640 1 --2- 172.21.15.19:0/1728744532 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f32380a3bb0 0x7f32380a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3248009430 tx=0x7f324802a8c0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f3255aa7640 1 -- 172.21.15.19:0/1728744532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3250100e90 msgr2=0x7f3250104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f3255aa7640 1 --2- 172.21.15.19:0/1728744532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3250100e90 0x7f3250104e10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f324000c970 tx=0x7f324000ce30 comp rx=0 tx=0).stop 2024-04-25T12:07:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f3255aa7640 1 -- 172.21.15.19:0/1728744532 shutdown_connections 2024-04-25T12:07:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f3255aa7640 1 --2- 172.21.15.19:0/1728744532 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f32380a3bb0 0x7f32380a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f3255aa7640 1 --2- 172.21.15.19:0/1728744532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32501029a0 0x7f3250103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f3255aa7640 1 --2- 172.21.15.19:0/1728744532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3250101fe0 0x7f3250103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f3255aa7640 1 --2- 172.21.15.19:0/1728744532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3250100e90 0x7f3250104e10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.958+0000 7f3255aa7640 1 -- 172.21.15.19:0/1728744532 >> 172.21.15.19:0/1728744532 conn(0x7f32500fc400 msgr2=0x7f32500fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.959+0000 7f3255aa7640 1 -- 172.21.15.19:0/1728744532 shutdown_connections 2024-04-25T12:07:00.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:00.959+0000 7f3255aa7640 1 -- 172.21.15.19:0/1728744532 wait complete. 2024-04-25T12:07:00.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:00 smithi019 ceph-mon[29213]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:07:00.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:00 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/324282005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:07:00.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:00 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3739161921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:07:00.986 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T12:07:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.092+0000 7f9f54c31640 1 -- 172.21.15.19:0/3627526080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f501031b0 msgr2=0x7f9f50103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.092+0000 7f9f54c31640 1 --2- 172.21.15.19:0/3627526080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f501031b0 0x7f9f50103630 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009900 tx=0x7f9f4402a410 comp rx=0 tx=0).stop 2024-04-25T12:07:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.092+0000 7f9f54c31640 1 -- 172.21.15.19:0/3627526080 shutdown_connections 2024-04-25T12:07:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.092+0000 7f9f54c31640 1 --2- 172.21.15.19:0/3627526080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f50103b70 0x7f9f5010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.092+0000 7f9f54c31640 1 --2- 172.21.15.19:0/3627526080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f501031b0 0x7f9f50103630 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.092+0000 7f9f54c31640 1 --2- 172.21.15.19:0/3627526080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f50101fc0 0x7f9f501023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.092+0000 7f9f54c31640 1 -- 172.21.15.19:0/3627526080 >> 172.21.15.19:0/3627526080 conn(0x7f9f500fd530 msgr2=0x7f9f500ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:01.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.092+0000 7f9f54c31640 1 -- 172.21.15.19:0/3627526080 shutdown_connections 2024-04-25T12:07:01.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.092+0000 7f9f54c31640 1 -- 172.21.15.19:0/3627526080 wait complete. 2024-04-25T12:07:01.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.093+0000 7f9f54c31640 1 Processor -- start 2024-04-25T12:07:01.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.093+0000 7f9f54c31640 1 -- start start 2024-04-25T12:07:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.093+0000 7f9f54c31640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f50101fc0 0x7f9f50198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.093+0000 7f9f54c31640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f501031b0 0x7f9f501987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.093+0000 7f9f54c31640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f50103b70 0x7f9f5019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.093+0000 7f9f54c31640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5010dc20 con 0x7f9f50103b70 2024-04-25T12:07:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.093+0000 7f9f54c31640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5010daa0 con 0x7f9f501031b0 2024-04-25T12:07:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.093+0000 7f9f54c31640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5010dda0 con 0x7f9f50101fc0 2024-04-25T12:07:01.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.093+0000 7f9f4ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f50103b70 0x7f9f5019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.093+0000 7f9f4ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f50103b70 0x7f9f5019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55636/0 (socket says 172.21.15.19:55636) 2024-04-25T12:07:01.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.093+0000 7f9f4ed9d640 1 -- 172.21.15.19:0/247403107 learned_addr learned my addr 172.21.15.19:0/247403107 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:01.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.093+0000 7f9f4e59c640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f50101fc0 0x7f9f50198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.093+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f501031b0 0x7f9f501987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.093+0000 7f9f4ed9d640 1 -- 172.21.15.19:0/247403107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f50101fc0 msgr2=0x7f9f50198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.094+0000 7f9f4ed9d640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f50101fc0 0x7f9f50198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.094+0000 7f9f4ed9d640 1 -- 172.21.15.19:0/247403107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f501031b0 msgr2=0x7f9f501987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.094+0000 7f9f4ed9d640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f501031b0 0x7f9f501987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.094+0000 7f9f4ed9d640 1 -- 172.21.15.19:0/247403107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5019fe80 con 0x7f9f50103b70 2024-04-25T12:07:01.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.094+0000 7f9f4e59c640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f50101fc0 0x7f9f50198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:07:01.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.094+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f501031b0 0x7f9f501987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:07:01.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.094+0000 7f9f4ed9d640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f50103b70 0x7f9f5019f810 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000eaa0 tx=0x7f9f4000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:01.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.094+0000 7f9f3f7fe640 1 -- 172.21.15.19:0/247403107 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000cdb0 con 0x7f9f50103b70 2024-04-25T12:07:01.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.094+0000 7f9f3f7fe640 1 -- 172.21.15.19:0/247403107 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9f40005020 con 0x7f9f50103b70 2024-04-25T12:07:01.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.094+0000 7f9f54c31640 1 -- 172.21.15.19:0/247403107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f501a0170 con 0x7f9f50103b70 2024-04-25T12:07:01.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.094+0000 7f9f3f7fe640 1 -- 172.21.15.19:0/247403107 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f400116b0 con 0x7f9f50103b70 2024-04-25T12:07:01.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.094+0000 7f9f54c31640 1 -- 172.21.15.19:0/247403107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f501a06c0 con 0x7f9f50103b70 2024-04-25T12:07:01.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.096+0000 7f9f3f7fe640 1 -- 172.21.15.19:0/247403107 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9f400051d0 con 0x7f9f50103b70 2024-04-25T12:07:01.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.096+0000 7f9f57a27740 1 Processor -- start 2024-04-25T12:07:01.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.096+0000 7f9f57a27740 1 -- start start 2024-04-25T12:07:01.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f57a27740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb2b80 0x557152cb2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f57a27740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb3d50 0x557152cb41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f57a27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb4710 0x557152cbafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f57a27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557152c17c50 con 0x557152cb4710 2024-04-25T12:07:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f57a27740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557152bb4830 con 0x557152cb2b80 2024-04-25T12:07:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f57a27740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557152c13c10 con 0x557152cb3d50 2024-04-25T12:07:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f3f7fe640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9f380a3bb0 0x7f9f380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f4dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb4710 0x557152cbafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f4dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb4710 0x557152cbafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55646/0 (socket says 172.21.15.19:55646) 2024-04-25T12:07:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f4dd9b640 1 -- 172.21.15.19:0/3179511051 learned_addr learned my addr 172.21.15.19:0/3179511051 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/3179511051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb2b80 0x557152cb2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f4ed9d640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9f380a3bb0 0x7f9f380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f4e59c640 1 --2- 172.21.15.19:0/3179511051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb3d50 0x557152cb41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f3f7fe640 1 -- 172.21.15.19:0/247403107 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9f400d06c0 con 0x7f9f50103b70 2024-04-25T12:07:01.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f4dd9b640 1 -- 172.21.15.19:0/3179511051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb3d50 msgr2=0x557152cb41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:01.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/3179511051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb3d50 0x557152cb41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f4dd9b640 1 -- 172.21.15.19:0/3179511051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb2b80 msgr2=0x557152cb2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:01.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/3179511051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb2b80 0x557152cb2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f4dd9b640 1 -- 172.21.15.19:0/3179511051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55715292a2d0 con 0x557152cb4710 2024-04-25T12:07:01.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/3179511051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb4710 0x557152cbafa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9f440057c0 tx=0x7f9f44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63ad891a9d8ccda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:01.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.097+0000 7f9f4ed9d640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9f380a3bb0 0x7f9f380a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f400ce510 tx=0x7f9f40066fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:01.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.098+0000 7f9f3cff9640 1 -- 172.21.15.19:0/3179511051 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44035040 con 0x557152cb4710 2024-04-25T12:07:01.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.098+0000 7f9f3cff9640 1 -- 172.21.15.19:0/3179511051 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9f44031b40 con 0x557152cb4710 2024-04-25T12:07:01.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.098+0000 7f9f3cff9640 1 -- 172.21.15.19:0/3179511051 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44003930 con 0x557152cb4710 2024-04-25T12:07:01.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.098+0000 7f9f57a27740 1 -- 172.21.15.19:0/3179511051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb4710 msgr2=0x557152cbafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.098+0000 7f9f57a27740 1 --2- 172.21.15.19:0/3179511051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb4710 0x557152cbafa0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9f440057c0 tx=0x7f9f44000bc0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.098+0000 7f9f57a27740 1 -- 172.21.15.19:0/3179511051 shutdown_connections 2024-04-25T12:07:01.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.098+0000 7f9f57a27740 1 --2- 172.21.15.19:0/3179511051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb4710 0x557152cbafa0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.098+0000 7f9f57a27740 1 --2- 172.21.15.19:0/3179511051 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb3d50 0x557152cb41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.098+0000 7f9f57a27740 1 --2- 172.21.15.19:0/3179511051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb2b80 0x557152cb2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.098+0000 7f9f57a27740 1 -- 172.21.15.19:0/3179511051 >> 172.21.15.19:0/3179511051 conn(0x557152cae070 msgr2=0x557152cae470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.098+0000 7f9f57a27740 1 -- 172.21.15.19:0/3179511051 shutdown_connections 2024-04-25T12:07:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f57a27740 1 -- 172.21.15.19:0/3179511051 wait complete. 2024-04-25T12:07:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f57a27740 1 -- start start 2024-04-25T12:07:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f57a27740 1 Processor -- start 2024-04-25T12:07:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f57a27740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb2b80 0x557152cbcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f57a27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb3d50 0x557152cbd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f57a27740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb4710 0x557152cc4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f57a27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557152cbb6c0 con 0x557152cb3d50 2024-04-25T12:07:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f57a27740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557152bb2470 con 0x557152cb2b80 2024-04-25T12:07:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f57a27740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557152cbb840 con 0x557152cb4710 2024-04-25T12:07:01.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f57a27740 10 client.0 _ll_register_callbacks cb 0x557152bd7f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:07:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f4e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb3d50 0x557152cbd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f4e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb3d50 0x557152cbd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55658/0 (socket says 172.21.15.19:55658) 2024-04-25T12:07:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f4e59c640 1 -- 172.21.15.19:0/147491651 learned_addr learned my addr 172.21.15.19:0/147491651 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/147491651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb2b80 0x557152cbcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/147491651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb4710 0x557152cc4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f4e59c640 1 -- 172.21.15.19:0/147491651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb4710 msgr2=0x557152cc4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f4e59c640 1 --2- 172.21.15.19:0/147491651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb4710 0x557152cc4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f4e59c640 1 -- 172.21.15.19:0/147491651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb2b80 msgr2=0x557152cbcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f4e59c640 1 --2- 172.21.15.19:0/147491651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb2b80 0x557152cbcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.099+0000 7f9f4e59c640 1 -- 172.21.15.19:0/147491651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571529f46b0 con 0x557152cb3d50 2024-04-25T12:07:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f4e59c640 1 --2- 172.21.15.19:0/147491651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb3d50 0x557152cbd4c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9f48013790 tx=0x7f9f48013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9689001e9eac0a45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:07:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f35ffb640 1 -- 172.21.15.19:0/147491651 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f48011cf0 con 0x557152cb3d50 2024-04-25T12:07:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f35ffb640 1 -- 172.21.15.19:0/147491651 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9f48004030 con 0x557152cb3d50 2024-04-25T12:07:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f57a27740 1 -- 172.21.15.19:0/147491651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f44031b40 con 0x557152cb3d50 2024-04-25T12:07:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f35ffb640 1 -- 172.21.15.19:0/147491651 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f48011e90 con 0x557152cb3d50 2024-04-25T12:07:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f57a27740 10 client.15351 fetch_fsmap learned FSMap version 26 2024-04-25T12:07:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f57a27740 1 -- 172.21.15.19:0/147491651 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557152cc7940 con 0x557152cb3d50 2024-04-25T12:07:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f35ffb640 1 -- 172.21.15.19:0/147491651 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9f44031b40 con 0x557152cb3d50 2024-04-25T12:07:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f35ffb640 1 -- 172.21.15.19:0/147491651 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9f4801a070 con 0x557152cb3d50 2024-04-25T12:07:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f57a27740 10 client.15351 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:07:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f57a27740 10 client.15351 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T12:07:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f57a27740 20 client.15351 populate_metadata read hostname 'smithi019' 2024-04-25T12:07:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f57a27740 1 --2- 172.21.15.19:0/147491651 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557152cc9fc0 0x557152ccb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f57a27740 4 client.15351 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T12:07:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.100+0000 7f9f57a27740 1 -- 172.21.15.19:0/147491651 --> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557152c25670 con 0x557152cc9fc0 2024-04-25T12:07:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.101+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/147491651 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557152cc9fc0 0x557152ccb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.101+0000 7f9f35ffb640 10 client.15351 ms_handle_connect on v2:172.21.15.169:6838/1723439500 2024-04-25T12:07:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.101+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/147491651 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557152cc9fc0 0x557152ccb430 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce45ac1747508c9 server_cookie=76f37cbb7b37c2f5 in_seq=0 out_seq=0 2024-04-25T12:07:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f35ffb640 1 -- 172.21.15.19:0/147491651 <== mds.0 v2:172.21.15.169:6838/1723439500 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557152c25670 con 0x557152cc9fc0 2024-04-25T12:07:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f35ffb640 10 client.15351 handle_command_reply: tid=0 2024-04-25T12:07:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f57a27740 1 client.15351 shutdown 2024-04-25T12:07:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f57a27740 1 -- 172.21.15.19:0/147491651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb3d50 msgr2=0x557152cbd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f57a27740 1 --2- 172.21.15.19:0/147491651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb3d50 0x557152cbd4c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9f48013790 tx=0x7f9f48013c50 comp rx=0 tx=0).stop 2024-04-25T12:07:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f57a27740 1 -- 172.21.15.19:0/147491651 shutdown_connections 2024-04-25T12:07:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f57a27740 1 --2- 172.21.15.19:0/147491651 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557152cc9fc0 0x557152ccb430 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f57a27740 1 --2- 172.21.15.19:0/147491651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb4710 0x557152cc4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f57a27740 1 --2- 172.21.15.19:0/147491651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb3d50 0x557152cbd4c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f35ffb640 0 client.15351 ms_handle_reset on v2:172.21.15.169:6838/1723439500 2024-04-25T12:07:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f57a27740 1 --2- 172.21.15.19:0/147491651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb2b80 0x557152cbcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f57a27740 1 -- 172.21.15.19:0/147491651 >> 172.21.15.19:0/147491651 conn(0x557152cae070 msgr2=0x557152cad1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f57a27740 1 -- 172.21.15.19:0/147491651 shutdown_connections 2024-04-25T12:07:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f57a27740 1 -- 172.21.15.19:0/147491651 wait complete. 2024-04-25T12:07:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.102+0000 7f9f57a27740 20 client.15351 trim_cache size 0 max 16384 2024-04-25T12:07:01.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.120+0000 7f9f57a27740 1 Processor -- start 2024-04-25T12:07:01.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.120+0000 7f9f57a27740 1 -- start start 2024-04-25T12:07:01.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f57a27740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb2b80 0x557152cbcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f57a27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb3d50 0x557152cbd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f57a27740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb4710 0x557152cb0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f57a27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557152cc9770 con 0x557152cb3d50 2024-04-25T12:07:01.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f57a27740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557152cc9e40 con 0x557152cb2b80 2024-04-25T12:07:01.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f57a27740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557152cca510 con 0x557152cb4710 2024-04-25T12:07:01.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f4e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb3d50 0x557152cbd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f4e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb3d50 0x557152cbd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55674/0 (socket says 172.21.15.19:55674) 2024-04-25T12:07:01.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f4e59c640 1 -- 172.21.15.19:0/4072612300 learned_addr learned my addr 172.21.15.19:0/4072612300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:01.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/4072612300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb4710 0x557152cb0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f4e59c640 1 -- 172.21.15.19:0/4072612300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb4710 msgr2=0x557152cb0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:01.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f4e59c640 1 --2- 172.21.15.19:0/4072612300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb4710 0x557152cb0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f4e59c640 1 -- 172.21.15.19:0/4072612300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb2b80 msgr2=0x557152cbcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:01.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f4e59c640 1 --2- 172.21.15.19:0/4072612300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb2b80 0x557152cbcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f4e59c640 1 -- 172.21.15.19:0/4072612300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557152cb37d0 con 0x557152cb3d50 2024-04-25T12:07:01.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/4072612300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb4710 0x557152cb0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:07:01.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f4e59c640 1 --2- 172.21.15.19:0/4072612300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb3d50 0x557152cbd0d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9f48004b90 tx=0x7f9f480096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8abcab88a7b29f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:01.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f37fff640 1 -- 172.21.15.19:0/4072612300 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f480037c0 con 0x557152cb3d50 2024-04-25T12:07:01.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f37fff640 1 -- 172.21.15.19:0/4072612300 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9f48003e30 con 0x557152cb3d50 2024-04-25T12:07:01.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.121+0000 7f9f37fff640 1 -- 172.21.15.19:0/4072612300 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f480056a0 con 0x557152cb3d50 2024-04-25T12:07:01.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 -- 172.21.15.19:0/4072612300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb3d50 msgr2=0x557152cbd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 --2- 172.21.15.19:0/4072612300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb3d50 0x557152cbd0d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9f48004b90 tx=0x7f9f480096a0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 -- 172.21.15.19:0/4072612300 shutdown_connections 2024-04-25T12:07:01.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 --2- 172.21.15.19:0/4072612300 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb4710 0x557152cb0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 --2- 172.21.15.19:0/4072612300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb3d50 0x557152cbd0d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 --2- 172.21.15.19:0/4072612300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb2b80 0x557152cbcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 -- 172.21.15.19:0/4072612300 >> 172.21.15.19:0/4072612300 conn(0x557152cae070 msgr2=0x557152ca90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:01.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 -- 172.21.15.19:0/4072612300 shutdown_connections 2024-04-25T12:07:01.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 -- 172.21.15.19:0/4072612300 wait complete. 2024-04-25T12:07:01.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 -- start start 2024-04-25T12:07:01.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 Processor -- start 2024-04-25T12:07:01.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb2b80 0x557152d2c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb3d50 0x557152d2c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb4710 0x557152d334a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557152ccb2b0 con 0x557152cb2b80 2024-04-25T12:07:01.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557152ccabe0 con 0x557152cb4710 2024-04-25T12:07:01.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.122+0000 7f9f57a27740 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x557152ccb980 con 0x557152cb3d50 2024-04-25T12:07:01.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f57a27740 10 client.0 _ll_register_callbacks cb 0x7f9f480095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T12:07:01.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f4dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb2b80 0x557152d2c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f4dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb2b80 0x557152d2c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55680/0 (socket says 172.21.15.19:55680) 2024-04-25T12:07:01.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f4dd9b640 1 -- 172.21.15.19:0/3924994667 learned_addr learned my addr 172.21.15.19:0/3924994667 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:01.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f4e59c640 1 --2- 172.21.15.19:0/3924994667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb3d50 0x557152d2c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/3924994667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb4710 0x557152d334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f4dd9b640 1 -- 172.21.15.19:0/3924994667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb3d50 msgr2=0x557152d2c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/3924994667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb3d50 0x557152d2c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f4dd9b640 1 -- 172.21.15.19:0/3924994667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb4710 msgr2=0x557152d334a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T12:07:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/3924994667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb4710 0x557152d334a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f4dd9b640 1 -- 172.21.15.19:0/3924994667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557152d33a90 con 0x557152cb2b80 2024-04-25T12:07:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f4dd9b640 1 --2- 172.21.15.19:0/3924994667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb2b80 0x557152d2c1a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9f44000c00 tx=0x7f9f4402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3f271027418881 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-04-25T12:07:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f35ffb640 1 -- 172.21.15.19:0/3924994667 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44035040 con 0x557152cb2b80 2024-04-25T12:07:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f35ffb640 1 -- 172.21.15.19:0/3924994667 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9f44004300 con 0x557152cb2b80 2024-04-25T12:07:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f57a27740 1 -- 172.21.15.19:0/3924994667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f48003e30 con 0x557152cb2b80 2024-04-25T12:07:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f35ffb640 1 -- 172.21.15.19:0/3924994667 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f44005020 con 0x557152cb2b80 2024-04-25T12:07:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f57a27740 10 client.15357 fetch_fsmap learned FSMap version 26 2024-04-25T12:07:01.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.123+0000 7f9f57a27740 1 -- 172.21.15.19:0/3924994667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557152d34d00 con 0x557152cb2b80 2024-04-25T12:07:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.124+0000 7f9f35ffb640 1 -- 172.21.15.19:0/3924994667 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f9f48003e30 con 0x557152cb2b80 2024-04-25T12:07:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.124+0000 7f9f35ffb640 1 -- 172.21.15.19:0/3924994667 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9f4404a040 con 0x557152cb2b80 2024-04-25T12:07:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.124+0000 7f9f57a27740 10 client.15357 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T12:07:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.124+0000 7f9f57a27740 10 client.15357 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T12:07:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.124+0000 7f9f57a27740 20 client.15357 populate_metadata read hostname 'smithi019' 2024-04-25T12:07:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.124+0000 7f9f57a27740 1 --2- 172.21.15.19:0/3924994667 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557152d394b0 0x557152d358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.124+0000 7f9f57a27740 4 client.15357 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T12:07:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.124+0000 7f9f57a27740 1 -- 172.21.15.19:0/3924994667 --> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] -- command(tid 0: {"prefix": "status"}) -- 0x557152c1f4e0 con 0x557152d394b0 2024-04-25T12:07:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.124+0000 7f9f4e59c640 1 --2- 172.21.15.19:0/3924994667 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557152d394b0 0x557152d358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.124+0000 7f9f35ffb640 10 client.15357 ms_handle_connect on v2:172.21.15.169:6838/1723439500 2024-04-25T12:07:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.124+0000 7f9f4e59c640 1 --2- 172.21.15.19:0/3924994667 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557152d394b0 0x557152d358f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=739b8231fe94f748 server_cookie=b1e7b354fdc1193f in_seq=0 out_seq=0 2024-04-25T12:07:01.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f35ffb640 1 -- 172.21.15.19:0/3924994667 <== mds.0 v2:172.21.15.169:6838/1723439500 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x557152c1f4e0 con 0x557152d394b0 2024-04-25T12:07:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f35ffb640 10 client.15357 handle_command_reply: tid=0 2024-04-25T12:07:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f57a27740 1 client.15357 shutdown 2024-04-25T12:07:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f57a27740 1 -- 172.21.15.19:0/3924994667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb2b80 msgr2=0x557152d2c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f57a27740 1 --2- 172.21.15.19:0/3924994667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb2b80 0x557152d2c1a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9f44000c00 tx=0x7f9f4402a410 comp rx=0 tx=0).stop 2024-04-25T12:07:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f57a27740 1 -- 172.21.15.19:0/3924994667 shutdown_connections 2024-04-25T12:07:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f57a27740 1 --2- 172.21.15.19:0/3924994667 >> [v2:172.21.15.169:6838/1723439500,v1:172.21.15.169:6839/1723439500] conn(0x557152d394b0 0x557152d358f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f57a27740 1 --2- 172.21.15.19:0/3924994667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557152cb4710 0x557152d334a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f57a27740 1 --2- 172.21.15.19:0/3924994667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557152cb3d50 0x557152d2c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f57a27740 1 --2- 172.21.15.19:0/3924994667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557152cb2b80 0x557152d2c1a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f35ffb640 0 client.15357 ms_handle_reset on v2:172.21.15.169:6838/1723439500 2024-04-25T12:07:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f57a27740 1 -- 172.21.15.19:0/3924994667 >> 172.21.15.19:0/3924994667 conn(0x557152cae070 msgr2=0x557152cbcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f57a27740 1 -- 172.21.15.19:0/3924994667 shutdown_connections 2024-04-25T12:07:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f57a27740 1 -- 172.21.15.19:0/3924994667 wait complete. 2024-04-25T12:07:01.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f57a27740 20 client.15357 trim_cache size 0 max 16384 2024-04-25T12:07:01.132 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:07:01.132 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "be62e290-02fa-11ef-bc93-c7b262605968", 2024-04-25T12:07:01.132 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-04-25T12:07:01.132 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24514, 2024-04-25T12:07:01.132 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-04-25T12:07:01.132 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-04-25T12:07:01.132 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-04-25T12:07:01.132 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 116.449315402, 2024-04-25T12:07:01.132 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-04-25T12:07:01.132 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 87, 2024-04-25T12:07:01.133 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T12:07:01.133 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 131.985286852 2024-04-25T12:07:01.133 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:07:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f54c31640 1 -- 172.21.15.19:0/247403107 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9f380a3bb0 msgr2=0x7f9f380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.125+0000 7f9f54c31640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9f380a3bb0 0x7f9f380a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f400ce510 tx=0x7f9f40066fd0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.126+0000 7f9f54c31640 1 -- 172.21.15.19:0/247403107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f50103b70 msgr2=0x7f9f5019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.126+0000 7f9f54c31640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f50103b70 0x7f9f5019f810 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000eaa0 tx=0x7f9f4000ef60 comp rx=0 tx=0).stop 2024-04-25T12:07:01.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.126+0000 7f9f54c31640 1 -- 172.21.15.19:0/247403107 shutdown_connections 2024-04-25T12:07:01.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.126+0000 7f9f54c31640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f9f380a3bb0 0x7f9f380a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.126+0000 7f9f54c31640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f50103b70 0x7f9f5019f810 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.126+0000 7f9f54c31640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f501031b0 0x7f9f501987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.126+0000 7f9f54c31640 1 --2- 172.21.15.19:0/247403107 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f50101fc0 0x7f9f50198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.126+0000 7f9f54c31640 1 -- 172.21.15.19:0/247403107 >> 172.21.15.19:0/247403107 conn(0x7f9f500fd530 msgr2=0x7f9f501064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:01.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.126+0000 7f9f54c31640 1 -- 172.21.15.19:0/247403107 shutdown_connections 2024-04-25T12:07:01.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.126+0000 7f9f54c31640 1 -- 172.21.15.19:0/247403107 wait complete. 2024-04-25T12:07:01.135 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T12:07:01.145 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T12:07:01.145 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T12:07:01.146 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T12:07:01.146 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T12:07:01.146 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T12:07:01.147 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T12:07:01.147 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T12:07:01.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:00 smithi169 ceph-mon[32710]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:07:01.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:00 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/324282005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:07:01.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:00 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3739161921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:07:01.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:00 smithi150 ceph-mon[32250]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:07:01.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:00 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/324282005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:07:01.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:00 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3739161921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:07:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.250+0000 7fe914ea7640 1 -- 172.21.15.19:0/405973913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe910101fe0 msgr2=0x7fe910102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.250+0000 7fe914ea7640 1 --2- 172.21.15.19:0/405973913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe910101fe0 0x7fe910102400 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe900009900 tx=0x7fe90002a410 comp rx=0 tx=0).stop 2024-04-25T12:07:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.250+0000 7fe914ea7640 1 -- 172.21.15.19:0/405973913 shutdown_connections 2024-04-25T12:07:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.250+0000 7fe914ea7640 1 --2- 172.21.15.19:0/405973913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe910103b90 0x7fe91010a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.250+0000 7fe914ea7640 1 --2- 172.21.15.19:0/405973913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9101031d0 0x7fe910103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.250+0000 7fe914ea7640 1 --2- 172.21.15.19:0/405973913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe910101fe0 0x7fe910102400 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.250+0000 7fe914ea7640 1 -- 172.21.15.19:0/405973913 >> 172.21.15.19:0/405973913 conn(0x7fe9100fd570 msgr2=0x7fe9100ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.250+0000 7fe914ea7640 1 -- 172.21.15.19:0/405973913 shutdown_connections 2024-04-25T12:07:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.250+0000 7fe914ea7640 1 -- 172.21.15.19:0/405973913 wait complete. 2024-04-25T12:07:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe914ea7640 1 Processor -- start 2024-04-25T12:07:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe914ea7640 1 -- start start 2024-04-25T12:07:01.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe914ea7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe910101fe0 0x7fe910198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe914ea7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9101031d0 0x7fe910198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe914ea7640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe910103b90 0x7fe91019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe914ea7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe91010dc40 con 0x7fe910101fe0 2024-04-25T12:07:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe914ea7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe91010dac0 con 0x7fe9101031d0 2024-04-25T12:07:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe914ea7640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe91010ddc0 con 0x7fe910103b90 2024-04-25T12:07:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe90e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe910101fe0 0x7fe910198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe90e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe910101fe0 0x7fe910198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55688/0 (socket says 172.21.15.19:55688) 2024-04-25T12:07:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe90e59c640 1 -- 172.21.15.19:0/3916865757 learned_addr learned my addr 172.21.15.19:0/3916865757 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe907fff640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9101031d0 0x7fe910198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe90ed9d640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe910103b90 0x7fe91019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe90e59c640 1 -- 172.21.15.19:0/3916865757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe910103b90 msgr2=0x7fe91019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe90e59c640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe910103b90 0x7fe91019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe90e59c640 1 -- 172.21.15.19:0/3916865757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9101031d0 msgr2=0x7fe910198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe90e59c640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9101031d0 0x7fe910198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe90e59c640 1 -- 172.21.15.19:0/3916865757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91019fe50 con 0x7fe910101fe0 2024-04-25T12:07:01.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe907fff640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9101031d0 0x7fe910198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:07:01.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.251+0000 7fe90e59c640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe910101fe0 0x7fe910198230 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe900006010 tx=0x7fe9000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:01.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.252+0000 7fe9077fe640 1 -- 172.21.15.19:0/3916865757 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90003f030 con 0x7fe910101fe0 2024-04-25T12:07:01.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.252+0000 7fe914ea7640 1 -- 172.21.15.19:0/3916865757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9101a00e0 con 0x7fe910101fe0 2024-04-25T12:07:01.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.252+0000 7fe9077fe640 1 -- 172.21.15.19:0/3916865757 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe900003cd0 con 0x7fe910101fe0 2024-04-25T12:07:01.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.252+0000 7fe9077fe640 1 -- 172.21.15.19:0/3916865757 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe900035070 con 0x7fe910101fe0 2024-04-25T12:07:01.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.252+0000 7fe914ea7640 1 -- 172.21.15.19:0/3916865757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9101a05d0 con 0x7fe910101fe0 2024-04-25T12:07:01.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.253+0000 7fe914ea7640 1 -- 172.21.15.19:0/3916865757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe910109300 con 0x7fe910101fe0 2024-04-25T12:07:01.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.254+0000 7fe9077fe640 1 -- 172.21.15.19:0/3916865757 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe900042030 con 0x7fe910101fe0 2024-04-25T12:07:01.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.254+0000 7fe9077fe640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe8fc0a3c50 0x7fe8fc0a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.255+0000 7fe907fff640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe8fc0a3c50 0x7fe8fc0a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.255+0000 7fe9077fe640 1 -- 172.21.15.19:0/3916865757 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fe9000f0b70 con 0x7fe910101fe0 2024-04-25T12:07:01.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.255+0000 7fe907fff640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe8fc0a3c50 0x7fe8fc0a6110 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8005d00 tx=0x7fe8f8005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.259+0000 7fe9077fe640 1 -- 172.21.15.19:0/3916865757 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9000bd530 con 0x7fe910101fe0 2024-04-25T12:07:01.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.500+0000 7fe914ea7640 1 -- 172.21.15.19:0/3916865757 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fe910062b50 con 0x7fe910101fe0 2024-04-25T12:07:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.741+0000 7fe9077fe640 1 -- 172.21.15.19:0/3916865757 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fe9000bce30 con 0x7fe910101fe0 2024-04-25T12:07:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.744+0000 7fe914ea7640 1 -- 172.21.15.19:0/3916865757 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe8fc0a3c50 msgr2=0x7fe8fc0a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.744+0000 7fe914ea7640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe8fc0a3c50 0x7fe8fc0a6110 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8005d00 tx=0x7fe8f8005c70 comp rx=0 tx=0).stop 2024-04-25T12:07:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.744+0000 7fe914ea7640 1 -- 172.21.15.19:0/3916865757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe910101fe0 msgr2=0x7fe910198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.744+0000 7fe914ea7640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe910101fe0 0x7fe910198230 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe900006010 tx=0x7fe9000042f0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.744+0000 7fe914ea7640 1 -- 172.21.15.19:0/3916865757 shutdown_connections 2024-04-25T12:07:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.744+0000 7fe914ea7640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fe8fc0a3c50 0x7fe8fc0a6110 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.744+0000 7fe914ea7640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe910103b90 0x7fe91019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.744+0000 7fe914ea7640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9101031d0 0x7fe910198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.744+0000 7fe914ea7640 1 --2- 172.21.15.19:0/3916865757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe910101fe0 0x7fe910198230 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.744+0000 7fe914ea7640 1 -- 172.21.15.19:0/3916865757 >> 172.21.15.19:0/3916865757 conn(0x7fe9100fd570 msgr2=0x7fe9101064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:01.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.744+0000 7fe914ea7640 1 -- 172.21.15.19:0/3916865757 shutdown_connections 2024-04-25T12:07:01.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.744+0000 7fe914ea7640 1 -- 172.21.15.19:0/3916865757 wait complete. 2024-04-25T12:07:01.761 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T12:07:01.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.870+0000 7f4575a7c640 1 -- 172.21.15.19:0/1876713989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570102040 msgr2=0x7f45701024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.870+0000 7f4575a7c640 1 --2- 172.21.15.19:0/1876713989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570102040 0x7f45701024c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4564009900 tx=0x7f456402a410 comp rx=0 tx=0).stop 2024-04-25T12:07:01.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.870+0000 7f4575a7c640 1 -- 172.21.15.19:0/1876713989 shutdown_connections 2024-04-25T12:07:01.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.870+0000 7f4575a7c640 1 --2- 172.21.15.19:0/1876713989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4570102a00 0x7f4570109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.870+0000 7f4575a7c640 1 --2- 172.21.15.19:0/1876713989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570102040 0x7f45701024c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.870+0000 7f4575a7c640 1 --2- 172.21.15.19:0/1876713989 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4570100e60 0x7f4570101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.870+0000 7f4575a7c640 1 -- 172.21.15.19:0/1876713989 >> 172.21.15.19:0/1876713989 conn(0x7f45700fc420 msgr2=0x7f45700fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:01.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.870+0000 7f4575a7c640 1 -- 172.21.15.19:0/1876713989 shutdown_connections 2024-04-25T12:07:01.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.870+0000 7f4575a7c640 1 -- 172.21.15.19:0/1876713989 wait complete. 2024-04-25T12:07:01.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.871+0000 7f4575a7c640 1 Processor -- start 2024-04-25T12:07:01.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.871+0000 7f4575a7c640 1 -- start start 2024-04-25T12:07:01.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.871+0000 7f4575a7c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4570100e60 0x7f4570198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.871+0000 7f4575a7c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570102040 0x7f4570198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.871+0000 7f4575a7c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4570102a00 0x7f457019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.871+0000 7f4575a7c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f457010cb30 con 0x7f4570102040 2024-04-25T12:07:01.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.871+0000 7f4575a7c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f457010c9b0 con 0x7f4570102a00 2024-04-25T12:07:01.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.871+0000 7f4575a7c640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f457010ccb0 con 0x7f4570100e60 2024-04-25T12:07:01.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.871+0000 7f456effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570102040 0x7f4570198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.871+0000 7f456effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570102040 0x7f4570198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56228/0 (socket says 172.21.15.19:56228) 2024-04-25T12:07:01.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.871+0000 7f456effd640 1 -- 172.21.15.19:0/79529464 learned_addr learned my addr 172.21.15.19:0/79529464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:01.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.872+0000 7f456f7fe640 1 --2- 172.21.15.19:0/79529464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4570100e60 0x7f4570198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.872+0000 7f456ffff640 1 --2- 172.21.15.19:0/79529464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4570102a00 0x7f457019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:01.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.872+0000 7f456effd640 1 -- 172.21.15.19:0/79529464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4570100e60 msgr2=0x7f4570198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.872+0000 7f456effd640 1 --2- 172.21.15.19:0/79529464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4570100e60 0x7f4570198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.872+0000 7f456effd640 1 -- 172.21.15.19:0/79529464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4570102a00 msgr2=0x7f457019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:01.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.872+0000 7f456effd640 1 --2- 172.21.15.19:0/79529464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4570102a00 0x7f457019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:01.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.872+0000 7f456effd640 1 -- 172.21.15.19:0/79529464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457019fe50 con 0x7f4570102040 2024-04-25T12:07:01.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.872+0000 7f456f7fe640 1 --2- 172.21.15.19:0/79529464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4570100e60 0x7f4570198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:07:01.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.872+0000 7f456effd640 1 --2- 172.21.15.19:0/79529464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570102040 0x7f4570198770 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4564005940 tx=0x7f45640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:01.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.872+0000 7f456cff9640 1 -- 172.21.15.19:0/79529464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456403f030 con 0x7f4570102040 2024-04-25T12:07:01.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.872+0000 7f456cff9640 1 -- 172.21.15.19:0/79529464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f4564003cd0 con 0x7f4570102040 2024-04-25T12:07:01.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.872+0000 7f456cff9640 1 -- 172.21.15.19:0/79529464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564035070 con 0x7f4570102040 2024-04-25T12:07:01.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.872+0000 7f4575a7c640 1 -- 172.21.15.19:0/79529464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45701a00e0 con 0x7f4570102040 2024-04-25T12:07:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.872+0000 7f4575a7c640 1 -- 172.21.15.19:0/79529464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45701a05d0 con 0x7f4570102040 2024-04-25T12:07:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.873+0000 7f4575a7c640 1 -- 172.21.15.19:0/79529464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45701a0830 con 0x7f4570102040 2024-04-25T12:07:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.875+0000 7f456cff9640 1 -- 172.21.15.19:0/79529464 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f4564042030 con 0x7f4570102040 2024-04-25T12:07:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.876+0000 7f456cff9640 1 --2- 172.21.15.19:0/79529464 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f45580a3ca0 0x7f45580a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.876+0000 7f456f7fe640 1 -- 172.21.15.19:0/79529464 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f45580a3ca0 msgr2=0x7f45580a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2853466745 2024-04-25T12:07:01.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.876+0000 7f456cff9640 1 -- 172.21.15.19:0/79529464 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f45640f0bf0 con 0x7f4570102040 2024-04-25T12:07:01.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.876+0000 7f456f7fe640 1 --2- 172.21.15.19:0/79529464 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f45580a3ca0 0x7f45580a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:07:01.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:01.879+0000 7f456cff9640 1 -- 172.21.15.19:0/79529464 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45640bcf60 con 0x7f4570102040 2024-04-25T12:07:02.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:01 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3916865757' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T12:07:02.048 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:01 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: ignoring --setuser ceph since I am not root 2024-04-25T12:07:02.048 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:01 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: ignoring --setgroup ceph since I am not root 2024-04-25T12:07:02.048 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:01 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:01.932+0000 7f256c1e4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:07:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.067+0000 7f4575a7c640 1 -- 172.21.15.19:0/79529464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f45701a0f30 con 0x7f4570102040 2024-04-25T12:07:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.073+0000 7f456cff9640 1 -- 172.21.15.19:0/79529464 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f45640c1260 con 0x7f4570102040 2024-04-25T12:07:02.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.075+0000 7f4575a7c640 1 -- 172.21.15.19:0/79529464 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f45580a3ca0 msgr2=0x7f45580a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:07:02.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.075+0000 7f4575a7c640 1 --2- 172.21.15.19:0/79529464 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f45580a3ca0 0x7f45580a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.075+0000 7f4575a7c640 1 -- 172.21.15.19:0/79529464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570102040 msgr2=0x7f4570198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:02.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.075+0000 7f4575a7c640 1 --2- 172.21.15.19:0/79529464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570102040 0x7f4570198770 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4564005940 tx=0x7f45640042f0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.075+0000 7f4575a7c640 1 -- 172.21.15.19:0/79529464 shutdown_connections 2024-04-25T12:07:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.075+0000 7f4575a7c640 1 --2- 172.21.15.19:0/79529464 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f45580a3ca0 0x7f45580a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.075+0000 7f4575a7c640 1 --2- 172.21.15.19:0/79529464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4570102a00 0x7f457019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.075+0000 7f4575a7c640 1 --2- 172.21.15.19:0/79529464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4570102040 0x7f4570198770 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.075+0000 7f4575a7c640 1 --2- 172.21.15.19:0/79529464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4570100e60 0x7f4570198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.075+0000 7f4575a7c640 1 -- 172.21.15.19:0/79529464 >> 172.21.15.19:0/79529464 conn(0x7f45700fc420 msgr2=0x7f4570069600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.075+0000 7f4575a7c640 1 -- 172.21.15.19:0/79529464 shutdown_connections 2024-04-25T12:07:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.075+0000 7f4575a7c640 1 -- 172.21.15.19:0/79529464 wait complete. 2024-04-25T12:07:02.085 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T12:07:02.107 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:01 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3916865757' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T12:07:02.108 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:01 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: ignoring --setuser ceph since I am not root 2024-04-25T12:07:02.108 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:01 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: ignoring --setgroup ceph since I am not root 2024-04-25T12:07:02.108 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:01 smithi169 ceph-mgr[34744]: -- 172.21.15.169:0/3513891696 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5614d73345a0 con 0x5614d7308800 2024-04-25T12:07:02.108 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:01 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:01.922+0000 7f4e029d3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:07:02.108 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:02 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:02.108+0000 7f4e029d3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:07:02.108 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:01 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3916865757' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T12:07:02.109 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:01 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: ignoring --setuser ceph since I am not root 2024-04-25T12:07:02.109 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:01 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: ignoring --setgroup ceph since I am not root 2024-04-25T12:07:02.109 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:01 smithi150 ceph-mgr[33650]: -- 172.21.15.150:0/646709683 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55678cd3c5a0 con 0x55678cd10c00 2024-04-25T12:07:02.109 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:01 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:01.925+0000 7f95e6399200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T12:07:02.109 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:02 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:02.109+0000 7f95e6399200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:07:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.191+0000 7ff445620640 1 -- 172.21.15.19:0/2936401447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff440101200 msgr2=0x7ff440101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.191+0000 7ff445620640 1 --2- 172.21.15.19:0/2936401447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff440101200 0x7ff440101660 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff434009860 tx=0x7ff43402a3d0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.191+0000 7ff445620640 1 -- 172.21.15.19:0/2936401447 shutdown_connections 2024-04-25T12:07:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.191+0000 7ff445620640 1 --2- 172.21.15.19:0/2936401447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff440101ba0 0x7ff440102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.191+0000 7ff445620640 1 --2- 172.21.15.19:0/2936401447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff440101200 0x7ff440101660 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.191+0000 7ff445620640 1 --2- 172.21.15.19:0/2936401447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4401008a0 0x7ff440100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.191+0000 7ff445620640 1 -- 172.21.15.19:0/2936401447 >> 172.21.15.19:0/2936401447 conn(0x7ff4400fc420 msgr2=0x7ff4400fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.191+0000 7ff445620640 1 -- 172.21.15.19:0/2936401447 shutdown_connections 2024-04-25T12:07:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.191+0000 7ff445620640 1 -- 172.21.15.19:0/2936401447 wait complete. 2024-04-25T12:07:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.192+0000 7ff445620640 1 Processor -- start 2024-04-25T12:07:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.192+0000 7ff445620640 1 -- start start 2024-04-25T12:07:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.192+0000 7ff445620640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4401008a0 0x7ff44019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.192+0000 7ff445620640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff440101200 0x7ff44019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.192+0000 7ff445620640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff440101ba0 0x7ff4401a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.192+0000 7ff445620640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff44010fe10 con 0x7ff440101200 2024-04-25T12:07:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.192+0000 7ff445620640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff44010fc90 con 0x7ff440101ba0 2024-04-25T12:07:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.192+0000 7ff445620640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff44010ff90 con 0x7ff4401008a0 2024-04-25T12:07:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.192+0000 7ff43e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff440101200 0x7ff44019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.192+0000 7ff43e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff440101200 0x7ff44019a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56240/0 (socket says 172.21.15.19:56240) 2024-04-25T12:07:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.192+0000 7ff43e7fc640 1 -- 172.21.15.19:0/3554795312 learned_addr learned my addr 172.21.15.19:0/3554795312 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.192+0000 7ff43effd640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4401008a0 0x7ff44019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.192+0000 7ff43f7fe640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff440101ba0 0x7ff4401a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.193+0000 7ff43effd640 1 -- 172.21.15.19:0/3554795312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff440101ba0 msgr2=0x7ff4401a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.193+0000 7ff43effd640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff440101ba0 0x7ff4401a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.193+0000 7ff43effd640 1 -- 172.21.15.19:0/3554795312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff440101200 msgr2=0x7ff44019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.193+0000 7ff43effd640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff440101200 0x7ff44019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.193+0000 7ff43effd640 1 -- 172.21.15.19:0/3554795312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4401a2050 con 0x7ff4401008a0 2024-04-25T12:07:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.193+0000 7ff43e7fc640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff440101200 0x7ff44019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:07:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.193+0000 7ff43effd640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4401008a0 0x7ff44019a430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff43000e9f0 tx=0x7ff43000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.193+0000 7ff427fff640 1 -- 172.21.15.19:0/3554795312 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43000ccc0 con 0x7ff4401008a0 2024-04-25T12:07:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.193+0000 7ff445620640 1 -- 172.21.15.19:0/3554795312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4401a2340 con 0x7ff4401008a0 2024-04-25T12:07:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.193+0000 7ff445620640 1 -- 172.21.15.19:0/3554795312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4401a28e0 con 0x7ff4401008a0 2024-04-25T12:07:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.193+0000 7ff427fff640 1 -- 172.21.15.19:0/3554795312 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff430010900 con 0x7ff4401008a0 2024-04-25T12:07:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.193+0000 7ff427fff640 1 -- 172.21.15.19:0/3554795312 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430018850 con 0x7ff4401008a0 2024-04-25T12:07:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.194+0000 7ff445620640 1 -- 172.21.15.19:0/3554795312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff44010b5f0 con 0x7ff4401008a0 2024-04-25T12:07:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.195+0000 7ff427fff640 1 -- 172.21.15.19:0/3554795312 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7ff430010ab0 con 0x7ff4401008a0 2024-04-25T12:07:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.196+0000 7ff427fff640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff4280a3ca0 0x7ff4280a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.196+0000 7ff43e7fc640 1 -- 172.21.15.19:0/3554795312 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff4280a3ca0 msgr2=0x7ff4280a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2853466745 2024-04-25T12:07:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.196+0000 7ff427fff640 1 -- 172.21.15.19:0/3554795312 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff4300d09c0 con 0x7ff4401008a0 2024-04-25T12:07:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.196+0000 7ff43e7fc640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff4280a3ca0 0x7ff4280a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:07:02.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.204+0000 7ff427fff640 1 -- 172.21.15.19:0/3554795312 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff43009ccd0 con 0x7ff4401008a0 2024-04-25T12:07:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.393+0000 7ff445620640 1 -- 172.21.15.19:0/3554795312 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7ff44002d5c0 con 0x7ff4401008a0 2024-04-25T12:07:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.397+0000 7ff43e7fc640 1 -- 172.21.15.19:0/3554795312 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff4280a3ca0 msgr2=0x7ff4280a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2853466745 2024-04-25T12:07:02.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.397+0000 7ff43e7fc640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff4280a3ca0 0x7ff4280a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:07:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.400+0000 7ff427fff640 1 -- 172.21.15.19:0/3554795312 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7ff44002d5c0 con 0x7ff4401008a0 2024-04-25T12:07:02.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.402+0000 7ff445620640 1 -- 172.21.15.19:0/3554795312 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff4280a3ca0 msgr2=0x7ff4280a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:07:02.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.402+0000 7ff445620640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff4280a3ca0 0x7ff4280a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.402+0000 7ff445620640 1 -- 172.21.15.19:0/3554795312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4401008a0 msgr2=0x7ff44019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:02.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.402+0000 7ff445620640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4401008a0 0x7ff44019a430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff43000e9f0 tx=0x7ff43000eeb0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.402+0000 7ff445620640 1 -- 172.21.15.19:0/3554795312 shutdown_connections 2024-04-25T12:07:02.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.402+0000 7ff445620640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7ff4280a3ca0 0x7ff4280a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.402+0000 7ff445620640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff440101ba0 0x7ff4401a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.402+0000 7ff445620640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff440101200 0x7ff44019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.402+0000 7ff445620640 1 --2- 172.21.15.19:0/3554795312 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4401008a0 0x7ff44019a430 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:02.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.402+0000 7ff445620640 1 -- 172.21.15.19:0/3554795312 >> 172.21.15.19:0/3554795312 conn(0x7ff4400fc420 msgr2=0x7ff4400ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:02.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.402+0000 7ff445620640 1 -- 172.21.15.19:0/3554795312 shutdown_connections 2024-04-25T12:07:02.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:02.402+0000 7ff445620640 1 -- 172.21.15.19:0/3554795312 wait complete. 2024-04-25T12:07:02.406 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:02 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:02.215+0000 7f4e029d3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:07:02.411 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:02 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:02.127+0000 7f256c1e4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:07:02.411 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:02 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:02.238+0000 7f256c1e4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:07:02.412 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T12:07:02.437 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T12:07:02.437 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T12:07:02.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:07:02.437 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T12:07:02.438 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:02 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:02.212+0000 7f95e6399200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T12:07:02.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:07:02.453 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T12:07:02.519 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-04-25T12:07:02.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:07:02.521 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'rbytes': False, 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-25T12:07:02.521 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T12:07:02.521 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T12:07:02.522 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T12:07:02.742 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:02 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:02.565+0000 7f256c1e4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:07:02.745 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:02 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:02.656+0000 7f256c1e4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:07:02.819 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:07:02.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:02 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3916865757' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T12:07:02.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:02 smithi169 ceph-mon[32710]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T12:07:02.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:02 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/79529464' entity='client.admin' 2024-04-25T12:07:02.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:02 smithi169 ceph-mon[32710]: from='client.? ' entity='client.admin' 2024-04-25T12:07:02.907 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:02 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:02.527+0000 7f4e029d3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:07:02.907 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:02 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:02.612+0000 7f4e029d3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:07:02.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:02 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3916865757' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T12:07:02.913 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:02 smithi150 ceph-mon[32250]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T12:07:02.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:02 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/79529464' entity='client.admin' 2024-04-25T12:07:02.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:02 smithi150 ceph-mon[32250]: from='client.? ' entity='client.admin' 2024-04-25T12:07:02.914 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:02 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:02.522+0000 7f95e6399200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T12:07:02.914 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:02 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:02.607+0000 7f95e6399200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T12:07:03.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:02 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3916865757' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T12:07:03.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:02 smithi019 ceph-mon[29213]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T12:07:03.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:02 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/79529464' entity='client.admin' 2024-04-25T12:07:03.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:02 smithi019 ceph-mon[29213]: from='client.? ' entity='client.admin' 2024-04-25T12:07:04.132 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:03 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:03.794+0000 7f95e6399200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:07:04.132 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:03 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:03.962+0000 7f95e6399200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:07:04.133 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:04.133+0000 7f95e6399200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:07:04.144 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:03 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:03.803+0000 7f4e029d3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:07:04.144 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:03 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:03.973+0000 7f4e029d3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:07:04.283 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:03 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:03.932+0000 7f256c1e4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T12:07:04.283 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:04 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:04.110+0000 7f256c1e4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T12:07:04.406 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:04 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:04.145+0000 7f4e029d3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:07:04.548 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:04 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:04.291+0000 7f256c1e4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T12:07:04.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.680+0000 7f76a67e4700 1 -- 172.21.15.19:0/3567336010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a0101930 msgr2=0x7f76a0101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:04.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.680+0000 7f76a67e4700 1 --2- 172.21.15.19:0/3567336010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a0101930 0x7f76a0101db0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7694009990 tx=0x7f7694009ca0 comp rx=0 tx=0).stop 2024-04-25T12:07:04.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.680+0000 7f76a67e4700 1 -- 172.21.15.19:0/3567336010 shutdown_connections 2024-04-25T12:07:04.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.680+0000 7f76a67e4700 1 --2- 172.21.15.19:0/3567336010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76a01022f0 0x7f76a010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:04.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.680+0000 7f76a67e4700 1 --2- 172.21.15.19:0/3567336010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a0107980 0x7f76a0107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:04.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.680+0000 7f76a67e4700 1 --2- 172.21.15.19:0/3567336010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a0101930 0x7f76a0101db0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:04.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.680+0000 7f76a67e4700 1 -- 172.21.15.19:0/3567336010 >> 172.21.15.19:0/3567336010 conn(0x7f76a00788b0 msgr2=0x7f76a0078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:04.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.680+0000 7f76a67e4700 1 -- 172.21.15.19:0/3567336010 shutdown_connections 2024-04-25T12:07:04.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.680+0000 7f76a67e4700 1 -- 172.21.15.19:0/3567336010 wait complete. 2024-04-25T12:07:04.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.681+0000 7f76a67e4700 1 Processor -- start 2024-04-25T12:07:04.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.681+0000 7f76a67e4700 1 -- start start 2024-04-25T12:07:04.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f76a67e4700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76a0101930 0x7f76a0199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f76a67e4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a01022f0 0x7f76a0199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f76a67e4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a0107980 0x7f76a019da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f76a67e4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76a0111c30 con 0x7f76a01022f0 2024-04-25T12:07:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f76a67e4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f76a0111ab0 con 0x7f76a0107980 2024-04-25T12:07:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f76a67e4700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f76a0111db0 con 0x7f76a0101930 2024-04-25T12:07:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f769f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a01022f0 0x7f76a0199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f769f7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a01022f0 0x7f76a0199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56266/0 (socket says 172.21.15.19:56266) 2024-04-25T12:07:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f769f7fe700 1 -- 172.21.15.19:0/2661333143 learned_addr learned my addr 172.21.15.19:0/2661333143 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f769ffff700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76a0101930 0x7f76a0199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f76a4d81700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a0107980 0x7f76a019da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f769f7fe700 1 -- 172.21.15.19:0/2661333143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76a0101930 msgr2=0x7f76a0199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:04.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f769f7fe700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76a0101930 0x7f76a0199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f769f7fe700 1 -- 172.21.15.19:0/2661333143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a0107980 msgr2=0x7f76a019da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f769f7fe700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a0107980 0x7f76a019da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f769f7fe700 1 -- 172.21.15.19:0/2661333143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a019e160 con 0x7f76a01022f0 2024-04-25T12:07:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f769ffff700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76a0101930 0x7f76a0199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:07:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f76a4d81700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a0107980 0x7f76a019da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:07:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f769f7fe700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a01022f0 0x7f76a0199690 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f76940098e0 tx=0x7f76940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f769d7fa700 1 -- 172.21.15.19:0/2661333143 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694027070 con 0x7f76a01022f0 2024-04-25T12:07:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.682+0000 7f76a67e4700 1 -- 172.21.15.19:0/2661333143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a019e3f0 con 0x7f76a01022f0 2024-04-25T12:07:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.683+0000 7f769d7fa700 1 -- 172.21.15.19:0/2661333143 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f769400b610 con 0x7f76a01022f0 2024-04-25T12:07:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.683+0000 7f769d7fa700 1 -- 172.21.15.19:0/2661333143 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76940173f0 con 0x7f76a01022f0 2024-04-25T12:07:04.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.683+0000 7f76a67e4700 1 -- 172.21.15.19:0/2661333143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a01a5cd0 con 0x7f76a01022f0 2024-04-25T12:07:04.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.684+0000 7f76a67e4700 1 -- 172.21.15.19:0/2661333143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a0069730 con 0x7f76a01022f0 2024-04-25T12:07:04.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.685+0000 7f769d7fa700 1 -- 172.21.15.19:0/2661333143 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f769402a070 con 0x7f76a01022f0 2024-04-25T12:07:04.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.685+0000 7f769d7fa700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f76880a3c90 0x7f76880a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:04.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.685+0000 7f769ffff700 1 -- 172.21.15.19:0/2661333143 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f76880a3c90 msgr2=0x7f76880a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2853466745 2024-04-25T12:07:04.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.685+0000 7f769d7fa700 1 -- 172.21.15.19:0/2661333143 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f76940d1240 con 0x7f76a01022f0 2024-04-25T12:07:04.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.685+0000 7f769ffff700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f76880a3c90 0x7f76880a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:07:04.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.690+0000 7f769d7fa700 1 -- 172.21.15.19:0/2661333143 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f769405eff0 con 0x7f76a01022f0 2024-04-25T12:07:04.720 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:04.423+0000 7f95e6399200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:07:04.720 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:04.514+0000 7f95e6399200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:07:04.720 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:04.721+0000 7f95e6399200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:07:04.733 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:04 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:04.435+0000 7f4e029d3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:07:04.733 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:04 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:04.528+0000 7f4e029d3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:07:04.733 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:04 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:04.734+0000 7f4e029d3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:07:04.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.886+0000 7f769ffff700 1 -- 172.21.15.19:0/2661333143 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f76880a3c90 msgr2=0x7f76880a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2853466745 2024-04-25T12:07:04.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.886+0000 7f769ffff700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f76880a3c90 0x7f76880a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:07:04.885 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:04 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:04.602+0000 7f256c1e4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T12:07:04.885 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:04 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:04.706+0000 7f256c1e4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T12:07:04.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.946+0000 7f76a67e4700 1 -- 172.21.15.19:0/2661333143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f76a019a4b0 con 0x7f76a01022f0 2024-04-25T12:07:04.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.947+0000 7f769d7fa700 1 -- 172.21.15.19:0/2661333143 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f7694013070 con 0x7f76a01022f0 2024-04-25T12:07:04.945 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T12:07:04.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.949+0000 7f76a67e4700 1 -- 172.21.15.19:0/2661333143 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f76880a3c90 msgr2=0x7f76880a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:07:04.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.949+0000 7f76a67e4700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f76880a3c90 0x7f76880a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:04.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.949+0000 7f76a67e4700 1 -- 172.21.15.19:0/2661333143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a01022f0 msgr2=0x7f76a0199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:04.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.949+0000 7f76a67e4700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a01022f0 0x7f76a0199690 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f76940098e0 tx=0x7f76940047b0 comp rx=0 tx=0).stop 2024-04-25T12:07:04.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.950+0000 7f76a67e4700 1 -- 172.21.15.19:0/2661333143 shutdown_connections 2024-04-25T12:07:04.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.950+0000 7f76a67e4700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7f76880a3c90 0x7f76880a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:04.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.950+0000 7f76a67e4700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76a0107980 0x7f76a019da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:04.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.950+0000 7f76a67e4700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76a0101930 0x7f76a0199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:04.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.950+0000 7f76a67e4700 1 --2- 172.21.15.19:0/2661333143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76a01022f0 0x7f76a0199690 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:04.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.950+0000 7f76a67e4700 1 -- 172.21.15.19:0/2661333143 >> 172.21.15.19:0/2661333143 conn(0x7f76a00788b0 msgr2=0x7f76a006b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:04.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.950+0000 7f76a67e4700 1 -- 172.21.15.19:0/2661333143 shutdown_connections 2024-04-25T12:07:04.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:04.950+0000 7f76a67e4700 1 -- 172.21.15.19:0/2661333143 wait complete. 2024-04-25T12:07:04.997 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:04.915+0000 7f95e6399200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:07:04.997 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:04 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:04.929+0000 7f4e029d3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:07:05.298 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:04 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:04.920+0000 7f256c1e4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T12:07:05.299 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:05 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:05.127+0000 7f256c1e4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T12:07:05.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:04 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2661333143' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T12:07:05.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:04 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2661333143' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T12:07:05.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:04 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2661333143' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T12:07:05.683 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T12:07:05.683 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T12:07:05.683 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-04-25T12:07:05.683 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T12:07:05.683 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T12:07:05.683 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T12:07:05.683 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T12:07:05.683 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T12:07:05.683 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T12:07:05.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:07:05.684 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-04-25T12:07:05.700 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T12:07:05.700 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T12:07:05.700 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T12:07:05.700 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T12:07:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-04-25T12:07:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T12:07:05.701 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T12:07:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-04-25T12:07:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-04-25T12:07:05.701 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T12:07:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-04-25T12:07:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-04-25T12:07:05.701 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T12:07:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T12:07:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-04-25T12:07:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T12:07:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42095sec preferred_lft 42095sec 2024-04-25T12:07:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-04-25T12:07:05.702 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-04-25T12:07:05.702 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T12:07:05.702 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T12:07:05.702 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-04-25T12:07:05.702 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T12:07:05.702 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:07:05.703 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T12:07:05.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-04-25T12:07:05.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-04-25T12:07:05.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-04-25T12:07:05.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T12:07:05.703 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T12:07:05.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:07:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:07:05.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:07:05.887 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T12:07:05.903 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:05 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:05.761+0000 7f256c1e4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:07:05.906 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:05 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:05.532+0000 7f4e029d3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:07:05.913 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-04-25T12:07:05.914 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:05 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:05.518+0000 7f95e6399200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T12:07:05.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:07:05.915 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-04-25T12:07:05.986 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-04-25T12:07:05.986 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T12:07:05.986 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T12:07:05.986 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T12:07:05.986 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T12:07:05.987 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:07:05.987 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T12:07:05.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T12:07:05.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T12:07:05.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T12:07:05.987 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T12:07:06.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:07:06.233 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:05 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:05.938+0000 7f95e6399200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:07:06.233 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:06 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:06.034+0000 7f95e6399200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:07:06.233 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:06 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:06.234+0000 7f95e6399200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:07:06.252 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:05 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:05.953+0000 7f4e029d3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:07:06.252 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:06 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:06.050+0000 7f4e029d3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:07:06.252 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:06 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:06.253+0000 7f4e029d3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:07:06.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:07:06.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:07:06.271 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T12:07:06.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:07:06.287 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-04-25T12:07:06.342 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:07:06.342 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T12:07:06.342 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T12:07:06.342 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T12:07:06.342 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T12:07:06.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:07:06.444 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:06 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:06.231+0000 7f256c1e4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T12:07:06.444 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:06 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:06.337+0000 7f256c1e4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T12:07:06.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:07:06.462 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T12:07:06.462 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:07:06.462 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T12:07:06.462 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T12:07:06.462 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T12:07:06.462 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T12:07:06.462 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T12:07:06.463 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T12:07:06.463 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T12:07:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:07:06.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:07:06.625 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T12:07:06.625 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-04-25T12:07:06.625 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-04-25T12:07:06.625 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T12:07:06.625 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-04-25T12:07:06.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:07:06.695 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:06 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:06.559+0000 7f256c1e4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T12:07:06.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:07:06.704 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T12:07:06.704 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T12:07:06.720 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T12:07:06.720 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T12:07:06.720 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T12:07:06.777 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T12:07:06.777 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T12:07:06.777 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/19c17715-a266-4b73-9313-334711baa8c6 2024-04-25T12:07:06.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:07:06.778 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/19c17715-a266-4b73-9313-334711baa8c6 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-25T12:07:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-25T12:07:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-04-25T12:07:06.881 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-04-25T12:07:06.990 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-25T12:07:06.991 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-04-25T12:07:06.991 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-04-25T12:07:06.991 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.150:3300,172.21.15.169:3300" 2024-04-25T12:07:06.991 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:3300,172.21.15.150:3300,172.21.15.169:3300:/volumes/qa/sv_1/19c17715-a266-4b73-9313-334711baa8c6 2024-04-25T12:07:06.991 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=be62e290-02fa-11ef-bc93-c7b262605968" will pass to kernel 2024-04-25T12:07:06.992 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T12:07:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-04-25T12:07:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T12:07:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T12:07:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T12:07:06.994 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T12:07:06.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:07:06.994 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T12:07:07.020 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T12:07:07.097 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:06 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:06.761+0000 7f95e6399200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:07:07.097 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:06 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:06.854+0000 7f95e6399200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:07:07.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T12:07:07.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:07:07.116 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:06 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:06.779+0000 7f4e029d3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:07:07.116 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:06 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:06.872+0000 7f4e029d3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:07:07.118 INFO:teuthology.orchestra.run.smithi019.stdout:client25060 2024-04-25T12:07:07.119 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T12:07:07.119 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T12:07:07.136 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T12:07:07.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T12:07:07.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:07:07.235 INFO:teuthology.orchestra.run.smithi019.stdout:client25060 2024-04-25T12:07:07.236 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T12:07:07.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T12:07:07.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:07:07.295 INFO:teuthology.orchestra.run.smithi019.stdout:client25060 2024-04-25T12:07:07.295 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/be62e290-02fa-11ef-bc93-c7b262605968.client25060/status') 2024-04-25T12:07:07.298 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:07 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:07.129+0000 7f256c1e4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T12:07:07.298 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:07 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:07.231+0000 7f256c1e4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T12:07:07.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T12:07:07.355 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.25060 (3)172.21.15.19:0/3455245093 2024-04-25T12:07:07.355 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-04-25T12:07:07.356 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-04-25T12:07:07.356 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-04-25T12:07:07.356 INFO:teuthology.orchestra.run.smithi019.stderr:71 bytes copied, 2.4526e-05 s, 2.9 MB/s 2024-04-25T12:07:07.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T12:07:07.363 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T12:07:07.373 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T12:07:07.373 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T12:07:07.373 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-25T12:07:07.406 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:07.116+0000 7f4e029d3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:07:07.406 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:07.254+0000 7f4e029d3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:07:07.406 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:07.350+0000 7f4e029d3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:07:07.413 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:07 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:07.098+0000 7f95e6399200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:07:07.413 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:07 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:07.232+0000 7f95e6399200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:07:07.414 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:07 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:07.326+0000 7f95e6399200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:07:07.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.480+0000 7fee44b76640 1 -- 172.21.15.19:0/2609777049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40107d30 msgr2=0x7fee4010a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.480+0000 7fee44b76640 1 --2- 172.21.15.19:0/2609777049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40107d30 0x7fee4010a120 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fee38009ca0 tx=0x7fee3802a8c0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.480+0000 7fee44b76640 1 -- 172.21.15.19:0/2609777049 shutdown_connections 2024-04-25T12:07:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.480+0000 7fee44b76640 1 --2- 172.21.15.19:0/2609777049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40107d30 0x7fee4010a120 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.480+0000 7fee44b76640 1 --2- 172.21.15.19:0/2609777049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee40105400 0x7fee401077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.480+0000 7fee44b76640 1 --2- 172.21.15.19:0/2609777049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee40102ad0 0x7fee40104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.480+0000 7fee44b76640 1 -- 172.21.15.19:0/2609777049 >> 172.21.15.19:0/2609777049 conn(0x7fee400fc420 msgr2=0x7fee400fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.480+0000 7fee44b76640 1 -- 172.21.15.19:0/2609777049 shutdown_connections 2024-04-25T12:07:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.480+0000 7fee44b76640 1 -- 172.21.15.19:0/2609777049 wait complete. 2024-04-25T12:07:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.481+0000 7fee44b76640 1 Processor -- start 2024-04-25T12:07:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.481+0000 7fee44b76640 1 -- start start 2024-04-25T12:07:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.481+0000 7fee44b76640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee40102ad0 0x7fee401981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.481+0000 7fee44b76640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee40105400 0x7fee40198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.481+0000 7fee44b76640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40107d30 0x7fee4019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:07.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.481+0000 7fee44b76640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee4010dc30 con 0x7fee40107d30 2024-04-25T12:07:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.481+0000 7fee44b76640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fee4010dab0 con 0x7fee40102ad0 2024-04-25T12:07:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.481+0000 7fee44b76640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fee4010ddb0 con 0x7fee40105400 2024-04-25T12:07:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.481+0000 7fee3ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40107d30 0x7fee4019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.481+0000 7fee3ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40107d30 0x7fee4019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56294/0 (socket says 172.21.15.19:56294) 2024-04-25T12:07:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.481+0000 7fee3ed9d640 1 -- 172.21.15.19:0/1503307392 learned_addr learned my addr 172.21.15.19:0/1503307392 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.481+0000 7fee3dd9b640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee40105400 0x7fee40198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.481+0000 7fee3e59c640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee40102ad0 0x7fee401981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.482+0000 7fee3ed9d640 1 -- 172.21.15.19:0/1503307392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee40105400 msgr2=0x7fee40198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.482+0000 7fee3ed9d640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee40105400 0x7fee40198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.482+0000 7fee3ed9d640 1 -- 172.21.15.19:0/1503307392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee40102ad0 msgr2=0x7fee401981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.482+0000 7fee3ed9d640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee40102ad0 0x7fee401981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.482+0000 7fee3ed9d640 1 -- 172.21.15.19:0/1503307392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4019fe10 con 0x7fee40107d30 2024-04-25T12:07:07.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.482+0000 7fee3e59c640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee40102ad0 0x7fee401981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:07:07.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.482+0000 7fee3ed9d640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40107d30 0x7fee4019f7a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fee38009dc0 tx=0x7fee38031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:07.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.482+0000 7fee2f7fe640 1 -- 172.21.15.19:0/1503307392 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38003bf0 con 0x7fee40107d30 2024-04-25T12:07:07.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.482+0000 7fee44b76640 1 -- 172.21.15.19:0/1503307392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee401a00a0 con 0x7fee40107d30 2024-04-25T12:07:07.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.482+0000 7fee2f7fe640 1 -- 172.21.15.19:0/1503307392 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fee38003d90 con 0x7fee40107d30 2024-04-25T12:07:07.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.482+0000 7fee2f7fe640 1 -- 172.21.15.19:0/1503307392 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3802fce0 con 0x7fee40107d30 2024-04-25T12:07:07.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.482+0000 7fee44b76640 1 -- 172.21.15.19:0/1503307392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee401a0640 con 0x7fee40107d30 2024-04-25T12:07:07.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.483+0000 7fee44b76640 1 -- 172.21.15.19:0/1503307392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee400fdf10 con 0x7fee40107d30 2024-04-25T12:07:07.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.484+0000 7fee2f7fe640 1 -- 172.21.15.19:0/1503307392 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fee38004040 con 0x7fee40107d30 2024-04-25T12:07:07.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.485+0000 7fee2f7fe640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fee280a3ca0 0x7fee280a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:07.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.485+0000 7fee3e59c640 1 -- 172.21.15.19:0/1503307392 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fee280a3ca0 msgr2=0x7fee280a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2853466745 2024-04-25T12:07:07.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.485+0000 7fee2f7fe640 1 -- 172.21.15.19:0/1503307392 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fee380f1bc0 con 0x7fee40107d30 2024-04-25T12:07:07.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.485+0000 7fee3e59c640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fee280a3ca0 0x7fee280a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:07:07.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.489+0000 7fee2f7fe640 1 -- 172.21.15.19:0/1503307392 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee380be6a0 con 0x7fee40107d30 2024-04-25T12:07:07.645 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:07 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:07.502+0000 7f256c1e4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T12:07:07.645 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:07 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:07.646+0000 7f256c1e4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T12:07:07.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.686+0000 7fee3e59c640 1 -- 172.21.15.19:0/1503307392 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fee280a3ca0 msgr2=0x7fee280a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2853466745 2024-04-25T12:07:07.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.686+0000 7fee3e59c640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fee280a3ca0 0x7fee280a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:07:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.721+0000 7fee44b76640 1 -- 172.21.15.19:0/1503307392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fee4002d5c0 con 0x7fee40107d30 2024-04-25T12:07:07.745 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:07 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:07.442+0000 7f95e6399200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:07:07.745 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:07 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:07.746+0000 7f95e6399200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:07:07.768 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:07.465+0000 7f4e029d3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:07:07.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.773+0000 7fee2f7fe640 1 -- 172.21.15.19:0/1503307392 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7fee380bdfa0 con 0x7fee40107d30 2024-04-25T12:07:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.775+0000 7fee44b76640 1 -- 172.21.15.19:0/1503307392 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fee280a3ca0 msgr2=0x7fee280a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:07:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.775+0000 7fee44b76640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fee280a3ca0 0x7fee280a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.775+0000 7fee44b76640 1 -- 172.21.15.19:0/1503307392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40107d30 msgr2=0x7fee4019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.775+0000 7fee44b76640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40107d30 0x7fee4019f7a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fee38009dc0 tx=0x7fee38031840 comp rx=0 tx=0).stop 2024-04-25T12:07:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.775+0000 7fee44b76640 1 -- 172.21.15.19:0/1503307392 shutdown_connections 2024-04-25T12:07:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.775+0000 7fee44b76640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fee280a3ca0 0x7fee280a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.775+0000 7fee44b76640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee40107d30 0x7fee4019f7a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.775+0000 7fee44b76640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee40105400 0x7fee40198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.775+0000 7fee44b76640 1 --2- 172.21.15.19:0/1503307392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee40102ad0 0x7fee401981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.775+0000 7fee44b76640 1 -- 172.21.15.19:0/1503307392 >> 172.21.15.19:0/1503307392 conn(0x7fee400fc420 msgr2=0x7fee40103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.775+0000 7fee44b76640 1 -- 172.21.15.19:0/1503307392 shutdown_connections 2024-04-25T12:07:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.775+0000 7fee44b76640 1 -- 172.21.15.19:0/1503307392 wait complete. 2024-04-25T12:07:07.785 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T12:07:07.796 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T12:07:07.825 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T12:07:07.825 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T12:07:07.825 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T12:07:07.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.933+0000 7fb7a1289640 1 -- 172.21.15.19:0/3609491307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c105400 msgr2=0x7fb79c1077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:07.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.933+0000 7fb7a1289640 1 --2- 172.21.15.19:0/3609491307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c105400 0x7fb79c1077f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb790009900 tx=0x7fb79002a410 comp rx=0 tx=0).stop 2024-04-25T12:07:07.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.933+0000 7fb7a1289640 1 -- 172.21.15.19:0/3609491307 shutdown_connections 2024-04-25T12:07:07.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.933+0000 7fb7a1289640 1 --2- 172.21.15.19:0/3609491307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c107d30 0x7fb79c10a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.933+0000 7fb7a1289640 1 --2- 172.21.15.19:0/3609491307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c105400 0x7fb79c1077f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.933+0000 7fb7a1289640 1 --2- 172.21.15.19:0/3609491307 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb79c102ad0 0x7fb79c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.933+0000 7fb7a1289640 1 -- 172.21.15.19:0/3609491307 >> 172.21.15.19:0/3609491307 conn(0x7fb79c0fc440 msgr2=0x7fb79c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:07.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.934+0000 7fb7a1289640 1 -- 172.21.15.19:0/3609491307 shutdown_connections 2024-04-25T12:07:07.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.934+0000 7fb7a1289640 1 -- 172.21.15.19:0/3609491307 wait complete. 2024-04-25T12:07:07.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.934+0000 7fb7a1289640 1 Processor -- start 2024-04-25T12:07:07.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.934+0000 7fb7a1289640 1 -- start start 2024-04-25T12:07:07.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.934+0000 7fb7a1289640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb79c102ad0 0x7fb79c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:07.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.934+0000 7fb7a1289640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c105400 0x7fb79c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:07.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.934+0000 7fb7a1289640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c107d30 0x7fb79c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:07.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.934+0000 7fb7a1289640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c10dc30 con 0x7fb79c105400 2024-04-25T12:07:07.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.934+0000 7fb7a1289640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c10dab0 con 0x7fb79c107d30 2024-04-25T12:07:07.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb7a1289640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c10ddb0 con 0x7fb79c102ad0 2024-04-25T12:07:07.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb79a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c105400 0x7fb79c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:07.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb79a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c105400 0x7fb79c198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56316/0 (socket says 172.21.15.19:56316) 2024-04-25T12:07:07.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb79affd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb79c102ad0 0x7fb79c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:07.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb79a7fc640 1 -- 172.21.15.19:0/2020919593 learned_addr learned my addr 172.21.15.19:0/2020919593 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:07.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb79b7fe640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c107d30 0x7fb79c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:07.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb79a7fc640 1 -- 172.21.15.19:0/2020919593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb79c102ad0 msgr2=0x7fb79c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:07.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb79a7fc640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb79c102ad0 0x7fb79c198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb79a7fc640 1 -- 172.21.15.19:0/2020919593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c107d30 msgr2=0x7fb79c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:07.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb79a7fc640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c107d30 0x7fb79c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:07.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb79a7fc640 1 -- 172.21.15.19:0/2020919593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79c19fe50 con 0x7fb79c105400 2024-04-25T12:07:07.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb79b7fe640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb79c107d30 0x7fb79c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:07:07.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb79affd640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb79c102ad0 0x7fb79c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:07:07.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb79a7fc640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c105400 0x7fb79c198770 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb790006010 tx=0x7fb7900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:07.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb783fff640 1 -- 172.21.15.19:0/2020919593 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79003f030 con 0x7fb79c105400 2024-04-25T12:07:07.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb7a1289640 1 -- 172.21.15.19:0/2020919593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79c1a00e0 con 0x7fb79c105400 2024-04-25T12:07:07.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb783fff640 1 -- 172.21.15.19:0/2020919593 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb790003cd0 con 0x7fb79c105400 2024-04-25T12:07:07.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.935+0000 7fb783fff640 1 -- 172.21.15.19:0/2020919593 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790035070 con 0x7fb79c105400 2024-04-25T12:07:07.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.936+0000 7fb7a1289640 1 -- 172.21.15.19:0/2020919593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79c1a05d0 con 0x7fb79c105400 2024-04-25T12:07:07.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.937+0000 7fb7a1289640 1 -- 172.21.15.19:0/2020919593 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb79c0fdf30 con 0x7fb79c105400 2024-04-25T12:07:07.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.938+0000 7fb783fff640 1 -- 172.21.15.19:0/2020919593 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fb790042030 con 0x7fb79c105400 2024-04-25T12:07:07.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.939+0000 7fb783fff640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb7840a3ca0 0x7fb7840a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:07.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.939+0000 7fb79affd640 1 -- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb7840a3ca0 msgr2=0x7fb7840a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2853466745 2024-04-25T12:07:07.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.939+0000 7fb783fff640 1 -- 172.21.15.19:0/2020919593 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fb7900f08a0 con 0x7fb79c105400 2024-04-25T12:07:07.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.939+0000 7fb79affd640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb7840a3ca0 0x7fb7840a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T12:07:07.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:07.944+0000 7fb783fff640 1 -- 172.21.15.19:0/2020919593 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7900bcc10 con 0x7fb79c105400 2024-04-25T12:07:08.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:07 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1503307392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T12:07:08.048 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:07 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:07.750+0000 7f256c1e4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T12:07:08.048 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:07 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:07.877+0000 7f256c1e4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T12:07:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:08.140+0000 7fb79affd640 1 -- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb7840a3ca0 msgr2=0x7fb7840a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2853466745 2024-04-25T12:07:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:08.140+0000 7fb79affd640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb7840a3ca0 0x7fb7840a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T12:07:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:08.156+0000 7fb7a1289640 1 -- 172.21.15.19:0/2020919593 --> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fb79c062390 con 0x7fb7840a3ca0 2024-04-25T12:07:08.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:07 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1503307392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T12:07:08.156 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:07.769+0000 7f4e029d3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:07:08.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:07 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1503307392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T12:07:08.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:08.541+0000 7fb79affd640 1 -- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb7840a3ca0 msgr2=0x7fb7840a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2853466745 2024-04-25T12:07:08.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:08.541+0000 7fb79affd640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb7840a3ca0 0x7fb7840a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T12:07:08.548 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:08.197+0000 7f256c1e4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T12:07:08.656 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:08.374+0000 7f4e029d3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:07:08.656 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z[34740]: 2024-04-25T12:07:08.589+0000 7f4e029d3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:07:08.663 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:08.351+0000 7f95e6399200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:07:08.664 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y[33646]: 2024-04-25T12:07:08.565+0000 7f95e6399200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:07:08.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:08.794+0000 7fb783fff640 1 -- 172.21.15.19:0/2020919593 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7fb7900b9280 con 0x7fb79c105400 2024-04-25T12:07:09.048 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:08.840+0000 7f256c1e4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T12:07:09.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1503307392' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T12:07:09.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-mon[29213]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T12:07:09.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-mon[29213]: Standby manager daemon y restarted 2024-04-25T12:07:09.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-mon[29213]: Standby manager daemon y started 2024-04-25T12:07:09.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.150:0/2363575379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T12:07:09.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.150:0/2363575379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:07:09.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.150:0/2363575379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T12:07:09.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.150:0/2363575379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:07:09.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-mon[29213]: Standby manager daemon z restarted 2024-04-25T12:07:09.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-mon[29213]: Standby manager daemon z started 2024-04-25T12:07:09.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.169:0/1772306436' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T12:07:09.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.169:0/1772306436' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:07:09.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.169:0/1772306436' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T12:07:09.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:08 smithi019 ceph-mon[29213]: from='mgr.? 172.21.15.169:0/1772306436' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:07:09.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:09.082+0000 7fb783fff640 1 -- 172.21.15.19:0/2020919593 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7fb7900b6730 con 0x7fb79c105400 2024-04-25T12:07:09.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:09.083+0000 7fb783fff640 1 -- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb7840a3ca0 msgr2=0x7fb7840a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T12:07:09.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:09.083+0000 7fb783fff640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:6800/2853466745,v1:172.21.15.19:6801/2853466745] conn(0x7fb7840a3ca0 0x7fb7840a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:09.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1503307392' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T12:07:09.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-mon[32710]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T12:07:09.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-mon[32710]: Standby manager daemon y restarted 2024-04-25T12:07:09.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-mon[32710]: Standby manager daemon y started 2024-04-25T12:07:09.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.150:0/2363575379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T12:07:09.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.150:0/2363575379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:07:09.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.150:0/2363575379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T12:07:09.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.150:0/2363575379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:07:09.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-mon[32710]: Standby manager daemon z restarted 2024-04-25T12:07:09.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-mon[32710]: Standby manager daemon z started 2024-04-25T12:07:09.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.169:0/1772306436' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T12:07:09.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.169:0/1772306436' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:07:09.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.169:0/1772306436' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T12:07:09.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:08 smithi169 ceph-mon[32710]: from='mgr.? 172.21.15.169:0/1772306436' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:07:09.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1503307392' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T12:07:09.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-mon[32250]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T12:07:09.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-mon[32250]: Standby manager daemon y restarted 2024-04-25T12:07:09.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-mon[32250]: Standby manager daemon y started 2024-04-25T12:07:09.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.150:0/2363575379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T12:07:09.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.150:0/2363575379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:07:09.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.150:0/2363575379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T12:07:09.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.150:0/2363575379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:07:09.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-mon[32250]: Standby manager daemon z restarted 2024-04-25T12:07:09.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-mon[32250]: Standby manager daemon z started 2024-04-25T12:07:09.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.169:0/1772306436' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T12:07:09.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.169:0/1772306436' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T12:07:09.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.169:0/1772306436' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T12:07:09.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:08 smithi150 ceph-mon[32250]: from='mgr.? 172.21.15.169:0/1772306436' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T12:07:09.338 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:09.070+0000 7f256c1e4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T12:07:09.339 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:09.313+0000 7f24cab8f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:07:09.339 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:09.313+0000 7f24cab8f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:07:09.339 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:09.313+0000 7f24cab8f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:07:09.339 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:09.313+0000 7f24cab8f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:07:09.339 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:09.313+0000 7f24cab8f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:07:09.339 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:09.314+0000 7f24cdb95700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:07:09.339 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:09.314+0000 7f24cdb95700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:07:09.339 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:09.314+0000 7f24cdb95700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:07:09.339 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:09.314+0000 7f24cdb95700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:07:09.339 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:07:09.314+0000 7f24cdb95700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:07:10.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-25T12:07:10.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: Active manager daemon x restarted 2024-04-25T12:07:10.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: Activating manager daemon x 2024-04-25T12:07:10.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T12:07:10.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: mgrmap e21: x(active, starting, since 0.0122016s), standbys: y, z 2024-04-25T12:07:10.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:07:10.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:07:10.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:07:10.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T12:07:10.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T12:07:10.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T12:07:10.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T12:07:10.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T12:07:10.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T12:07:10.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T12:07:10.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T12:07:10.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T12:07:10.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T12:07:10.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T12:07:10.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T12:07:10.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T12:07:10.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T12:07:10.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T12:07:10.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: Manager daemon x is now available 2024-04-25T12:07:10.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:07:10.052 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T12:07:10.052 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:09 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T12:07:10.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.089+0000 7fb783fff640 1 -- 172.21.15.19:0/2020919593 <== mon.0 v2:172.21.15.19:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fb790098f60 con 0x7fb79c105400 2024-04-25T12:07:10.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.090+0000 7fb783fff640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb784035660 0x7fb784035a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:10.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.090+0000 7fb783fff640 1 -- 172.21.15.19:0/2020919593 --> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fb79003ed60 con 0x7fb784035660 2024-04-25T12:07:10.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.091+0000 7fb79affd640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb784035660 0x7fb784035a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:10.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.093+0000 7fb79affd640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb784035660 0x7fb784035a40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009b80 tx=0x7fb78c005f90 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:10.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-25T12:07:10.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: Active manager daemon x restarted 2024-04-25T12:07:10.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: Activating manager daemon x 2024-04-25T12:07:10.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T12:07:10.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: mgrmap e21: x(active, starting, since 0.0122016s), standbys: y, z 2024-04-25T12:07:10.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:07:10.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:07:10.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:07:10.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T12:07:10.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T12:07:10.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T12:07:10.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T12:07:10.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T12:07:10.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T12:07:10.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T12:07:10.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T12:07:10.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T12:07:10.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T12:07:10.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T12:07:10.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T12:07:10.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T12:07:10.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T12:07:10.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T12:07:10.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:07:10.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:07:10.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:07:10.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:07:10.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:07:10.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:07:10.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:07:10.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:07:10.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:07:10.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:07:10.160 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:07:10.160 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:07:10.160 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T12:07:10.160 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T12:07:10.160 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T12:07:10.160 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: Manager daemon x is now available 2024-04-25T12:07:10.160 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:07:10.160 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T12:07:10.160 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:09 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T12:07:10.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.162+0000 7fb783fff640 1 -- 172.21.15.19:0/2020919593 <== mgr.15381 v2:172.21.15.19:6800/3657854393 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb79003ed60 con 0x7fb784035660 2024-04-25T12:07:10.161 INFO:teuthology.orchestra.run.smithi019.stdout:Schedule set for path / 2024-04-25T12:07:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.164+0000 7fb7a1289640 1 -- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb784035660 msgr2=0x7fb784035a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.164+0000 7fb7a1289640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb784035660 0x7fb784035a40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009b80 tx=0x7fb78c005f90 comp rx=0 tx=0).stop 2024-04-25T12:07:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.164+0000 7fb7a1289640 1 -- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c105400 msgr2=0x7fb79c198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.164+0000 7fb7a1289640 1 --2- 172.21.15.19:0/2020919593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c105400 0x7fb79c198770 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb790006010 tx=0x7fb7900042f0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.164+0000 7fb79b7fe640 1 -- 172.21.15.19:0/2020919593 reap_dead start 2024-04-25T12:07:10.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.164+0000 7fb7a1289640 1 -- 172.21.15.19:0/2020919593 shutdown_connections 2024-04-25T12:07:10.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.164+0000 7fb7a1289640 1 -- 172.21.15.19:0/2020919593 >> 172.21.15.19:0/2020919593 conn(0x7fb79c0fc440 msgr2=0x7fb79c103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:10.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.164+0000 7fb7a1289640 1 -- 172.21.15.19:0/2020919593 shutdown_connections 2024-04-25T12:07:10.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.164+0000 7fb7a1289640 1 -- 172.21.15.19:0/2020919593 wait complete. 2024-04-25T12:07:10.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-25T12:07:10.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: Active manager daemon x restarted 2024-04-25T12:07:10.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: Activating manager daemon x 2024-04-25T12:07:10.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T12:07:10.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: mgrmap e21: x(active, starting, since 0.0122016s), standbys: y, z 2024-04-25T12:07:10.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T12:07:10.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T12:07:10.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T12:07:10.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T12:07:10.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T12:07:10.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T12:07:10.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T12:07:10.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T12:07:10.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T12:07:10.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T12:07:10.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T12:07:10.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T12:07:10.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T12:07:10.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T12:07:10.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T12:07:10.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T12:07:10.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T12:07:10.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T12:07:10.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T12:07:10.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T12:07:10.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T12:07:10.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: Manager daemon x is now available 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T12:07:10.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:09 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T12:07:10.173 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T12:07:10.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.282+0000 7f19ec2da640 1 -- 172.21.15.19:0/1802295113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e4105360 msgr2=0x7f19e4107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:10.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.282+0000 7f19ec2da640 1 --2- 172.21.15.19:0/1802295113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e4105360 0x7f19e4107750 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009900 tx=0x7f19d402a410 comp rx=0 tx=0).stop 2024-04-25T12:07:10.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.282+0000 7f19ec2da640 1 -- 172.21.15.19:0/1802295113 shutdown_connections 2024-04-25T12:07:10.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.282+0000 7f19ec2da640 1 --2- 172.21.15.19:0/1802295113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e4107c90 0x7f19e410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.282+0000 7f19ec2da640 1 --2- 172.21.15.19:0/1802295113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e4105360 0x7f19e4107750 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.282+0000 7f19ec2da640 1 --2- 172.21.15.19:0/1802295113 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e4102a30 0x7f19e4104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.282+0000 7f19ec2da640 1 -- 172.21.15.19:0/1802295113 >> 172.21.15.19:0/1802295113 conn(0x7f19e40fc420 msgr2=0x7f19e40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:10.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.282+0000 7f19ec2da640 1 -- 172.21.15.19:0/1802295113 shutdown_connections 2024-04-25T12:07:10.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.282+0000 7f19ec2da640 1 -- 172.21.15.19:0/1802295113 wait complete. 2024-04-25T12:07:10.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.282+0000 7f19ec2da640 1 Processor -- start 2024-04-25T12:07:10.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.282+0000 7f19ec2da640 1 -- start start 2024-04-25T12:07:10.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ec2da640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e4102a30 0x7f19e4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:10.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ec2da640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e4105360 0x7f19e41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:10.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ec2da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e4107c90 0x7f19e419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:10.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ec2da640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19e410dc10 con 0x7f19e4107c90 2024-04-25T12:07:10.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ec2da640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f19e410da90 con 0x7f19e4102a30 2024-04-25T12:07:10.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ec2da640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f19e410dd90 con 0x7f19e4105360 2024-04-25T12:07:10.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ea877640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e4107c90 0x7f19e419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ea877640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e4107c90 0x7f19e419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56480/0 (socket says 172.21.15.19:56480) 2024-04-25T12:07:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ea877640 1 -- 172.21.15.19:0/1342376341 learned_addr learned my addr 172.21.15.19:0/1342376341 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ea076640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e4102a30 0x7f19e4198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19e9875640 1 --2- 172.21.15.19:0/1342376341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e4105360 0x7f19e41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ea877640 1 -- 172.21.15.19:0/1342376341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e4105360 msgr2=0x7f19e41987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ea877640 1 --2- 172.21.15.19:0/1342376341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e4105360 0x7f19e41987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ea877640 1 -- 172.21.15.19:0/1342376341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e4102a30 msgr2=0x7f19e4198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ea877640 1 --2- 172.21.15.19:0/1342376341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e4102a30 0x7f19e4198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ea877640 1 -- 172.21.15.19:0/1342376341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e419fe80 con 0x7f19e4107c90 2024-04-25T12:07:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ea076640 1 --2- 172.21.15.19:0/1342376341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e4102a30 0x7f19e4198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T12:07:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ea877640 1 --2- 172.21.15.19:0/1342376341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e4107c90 0x7f19e419f810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00e970 tx=0x7f19dc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19db7fe640 1 -- 172.21.15.19:0/1342376341 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00ccc0 con 0x7f19e4107c90 2024-04-25T12:07:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19db7fe640 1 -- 172.21.15.19:0/1342376341 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f19dc00ce60 con 0x7f19e4107c90 2024-04-25T12:07:10.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ec2da640 1 -- 172.21.15.19:0/1342376341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e41a0170 con 0x7f19e4107c90 2024-04-25T12:07:10.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19db7fe640 1 -- 172.21.15.19:0/1342376341 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc010430 con 0x7f19e4107c90 2024-04-25T12:07:10.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.283+0000 7f19ec2da640 1 -- 172.21.15.19:0/1342376341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e41a0640 con 0x7f19e4107c90 2024-04-25T12:07:10.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.284+0000 7f19ec2da640 1 -- 172.21.15.19:0/1342376341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e40fdf10 con 0x7f19e4107c90 2024-04-25T12:07:10.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.286+0000 7f19db7fe640 1 -- 172.21.15.19:0/1342376341 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f19dc005020 con 0x7f19e4107c90 2024-04-25T12:07:10.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.286+0000 7f19db7fe640 1 --2- 172.21.15.19:0/1342376341 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f19d00a3a60 0x7f19d00a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:10.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.287+0000 7f19ea076640 1 --2- 172.21.15.19:0/1342376341 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f19d00a3a60 0x7f19d00a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:10.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.287+0000 7f19db7fe640 1 -- 172.21.15.19:0/1342376341 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f19dc0d0550 con 0x7f19e4107c90 2024-04-25T12:07:10.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.287+0000 7f19ea076640 1 --2- 172.21.15.19:0/1342376341 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f19d00a3a60 0x7f19d00a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f19e0005fd0 tx=0x7f19e0009500 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:10.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.291+0000 7f19db7fe640 1 -- 172.21.15.19:0/1342376341 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19dc09d0c0 con 0x7f19e4107c90 2024-04-25T12:07:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.508+0000 7f19ec2da640 1 -- 172.21.15.19:0/1342376341 --> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f19e4062430 con 0x7f19d00a3a60 2024-04-25T12:07:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.563+0000 7f19db7fe640 1 -- 172.21.15.19:0/1342376341 <== mgr.15381 v2:172.21.15.19:6800/3657854393 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f19e4062430 con 0x7f19d00a3a60 2024-04-25T12:07:10.561 INFO:teuthology.orchestra.run.smithi019.stdout:Retention added to path / 2024-04-25T12:07:10.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.565+0000 7f19ec2da640 1 -- 172.21.15.19:0/1342376341 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f19d00a3a60 msgr2=0x7f19d00a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.565+0000 7f19ec2da640 1 --2- 172.21.15.19:0/1342376341 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f19d00a3a60 0x7f19d00a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f19e0005fd0 tx=0x7f19e0009500 comp rx=0 tx=0).stop 2024-04-25T12:07:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.565+0000 7f19ec2da640 1 -- 172.21.15.19:0/1342376341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e4107c90 msgr2=0x7f19e419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.565+0000 7f19ec2da640 1 --2- 172.21.15.19:0/1342376341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e4107c90 0x7f19e419f810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00e970 tx=0x7f19dc00ed70 comp rx=0 tx=0).stop 2024-04-25T12:07:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.565+0000 7f19ec2da640 1 -- 172.21.15.19:0/1342376341 shutdown_connections 2024-04-25T12:07:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.565+0000 7f19ec2da640 1 --2- 172.21.15.19:0/1342376341 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f19d00a3a60 0x7f19d00a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.565+0000 7f19ec2da640 1 --2- 172.21.15.19:0/1342376341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e4107c90 0x7f19e419f810 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.565+0000 7f19ec2da640 1 --2- 172.21.15.19:0/1342376341 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19e4105360 0x7f19e41987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.565+0000 7f19ec2da640 1 --2- 172.21.15.19:0/1342376341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19e4102a30 0x7f19e4198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.565+0000 7f19ec2da640 1 -- 172.21.15.19:0/1342376341 >> 172.21.15.19:0/1342376341 conn(0x7f19e40fc420 msgr2=0x7f19e4103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.565+0000 7f19ec2da640 1 -- 172.21.15.19:0/1342376341 shutdown_connections 2024-04-25T12:07:10.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.565+0000 7f19ec2da640 1 -- 172.21.15.19:0/1342376341 wait complete. 2024-04-25T12:07:10.575 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T12:07:10.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.681+0000 7fb23276f640 1 -- 172.21.15.19:0/3141136891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb22c103270 msgr2=0x7fb22c1036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:10.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.681+0000 7fb23276f640 1 --2- 172.21.15.19:0/3141136891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb22c103270 0x7fb22c1036f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb220009930 tx=0x7fb22002a420 comp rx=0 tx=0).stop 2024-04-25T12:07:10.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.681+0000 7fb23276f640 1 -- 172.21.15.19:0/3141136891 shutdown_connections 2024-04-25T12:07:10.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.681+0000 7fb23276f640 1 --2- 172.21.15.19:0/3141136891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb22c103c30 0x7fb22c10a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.681+0000 7fb23276f640 1 --2- 172.21.15.19:0/3141136891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb22c103270 0x7fb22c1036f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.681+0000 7fb23276f640 1 --2- 172.21.15.19:0/3141136891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb22c1002a0 0x7fb22c1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.681+0000 7fb23276f640 1 -- 172.21.15.19:0/3141136891 >> 172.21.15.19:0/3141136891 conn(0x7fb22c0fbbd0 msgr2=0x7fb22c0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:10.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.681+0000 7fb23276f640 1 -- 172.21.15.19:0/3141136891 shutdown_connections 2024-04-25T12:07:10.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.681+0000 7fb23276f640 1 -- 172.21.15.19:0/3141136891 wait complete. 2024-04-25T12:07:10.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.681+0000 7fb23276f640 1 Processor -- start 2024-04-25T12:07:10.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.681+0000 7fb23276f640 1 -- start start 2024-04-25T12:07:10.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb23276f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb22c1002a0 0x7fb22c102be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:10.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb23276f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb22c103270 0x7fb22c101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:10.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb23276f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb22c103c30 0x7fb22c101760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:10.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb23276f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c10dc60 con 0x7fb22c1002a0 2024-04-25T12:07:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb23276f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c10dae0 con 0x7fb22c103270 2024-04-25T12:07:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb23276f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c10dde0 con 0x7fb22c103c30 2024-04-25T12:07:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb22bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb22c1002a0 0x7fb22c102be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb22bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb22c1002a0 0x7fb22c102be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56496/0 (socket says 172.21.15.19:56496) 2024-04-25T12:07:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb22bfff640 1 -- 172.21.15.19:0/3350801801 learned_addr learned my addr 172.21.15.19:0/3350801801 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb22b7fe640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb22c103270 0x7fb22c101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb230d0c640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb22c103c30 0x7fb22c101760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb22bfff640 1 -- 172.21.15.19:0/3350801801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb22c103c30 msgr2=0x7fb22c101760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb22bfff640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb22c103c30 0x7fb22c101760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb22bfff640 1 -- 172.21.15.19:0/3350801801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb22c103270 msgr2=0x7fb22c101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb22bfff640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb22c103270 0x7fb22c101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb22bfff640 1 -- 172.21.15.19:0/3350801801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22c101ff0 con 0x7fb22c1002a0 2024-04-25T12:07:10.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb230d0c640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb22c103c30 0x7fb22c101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:07:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb22bfff640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb22c1002a0 0x7fb22c102be0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00e9c0 tx=0x7fb21c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb2297fa640 1 -- 172.21.15.19:0/3350801801 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c00cd80 con 0x7fb22c1002a0 2024-04-25T12:07:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb23276f640 1 -- 172.21.15.19:0/3350801801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22c1a2390 con 0x7fb22c1002a0 2024-04-25T12:07:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb2297fa640 1 -- 172.21.15.19:0/3350801801 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb21c0035f0 con 0x7fb22c1002a0 2024-04-25T12:07:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.682+0000 7fb2297fa640 1 -- 172.21.15.19:0/3350801801 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c010690 con 0x7fb22c1002a0 2024-04-25T12:07:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.683+0000 7fb23276f640 1 -- 172.21.15.19:0/3350801801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22c1a2860 con 0x7fb22c1002a0 2024-04-25T12:07:10.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.684+0000 7fb23276f640 1 -- 172.21.15.19:0/3350801801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb22c04fce0 con 0x7fb22c1002a0 2024-04-25T12:07:10.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.685+0000 7fb2297fa640 1 -- 172.21.15.19:0/3350801801 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fb21c003b30 con 0x7fb22c1002a0 2024-04-25T12:07:10.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.685+0000 7fb2297fa640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb2140a3a60 0x7fb2140a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:10.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.685+0000 7fb22b7fe640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb2140a3a60 0x7fb2140a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:10.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.685+0000 7fb2297fa640 1 -- 172.21.15.19:0/3350801801 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb21c0d05a0 con 0x7fb22c1002a0 2024-04-25T12:07:10.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.686+0000 7fb22b7fe640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb2140a3a60 0x7fb2140a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb220006010 tx=0x7fb220031510 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:10.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.690+0000 7fb2297fa640 1 -- 172.21.15.19:0/3350801801 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb21c09d150 con 0x7fb22c1002a0 2024-04-25T12:07:10.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.904+0000 7fb23276f640 1 -- 172.21.15.19:0/3350801801 --> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb22c02d9a0 con 0x7fb2140a3a60 2024-04-25T12:07:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.912+0000 7fb2297fa640 1 -- 172.21.15.19:0/3350801801 <== mgr.15381 v2:172.21.15.19:6800/3657854393 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fb22c02d9a0 con 0x7fb2140a3a60 2024-04-25T12:07:10.911 INFO:teuthology.orchestra.run.smithi019.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T12:07:10", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T12:07:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.915+0000 7fb23276f640 1 -- 172.21.15.19:0/3350801801 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb2140a3a60 msgr2=0x7fb2140a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.915+0000 7fb23276f640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb2140a3a60 0x7fb2140a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb220006010 tx=0x7fb220031510 comp rx=0 tx=0).stop 2024-04-25T12:07:10.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.915+0000 7fb23276f640 1 -- 172.21.15.19:0/3350801801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb22c1002a0 msgr2=0x7fb22c102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.915+0000 7fb23276f640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb22c1002a0 0x7fb22c102be0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00e9c0 tx=0x7fb21c00ee80 comp rx=0 tx=0).stop 2024-04-25T12:07:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.915+0000 7fb23276f640 1 -- 172.21.15.19:0/3350801801 shutdown_connections 2024-04-25T12:07:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.915+0000 7fb23276f640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb2140a3a60 0x7fb2140a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.915+0000 7fb23276f640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb22c103c30 0x7fb22c101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.915+0000 7fb23276f640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb22c103270 0x7fb22c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.915+0000 7fb23276f640 1 --2- 172.21.15.19:0/3350801801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb22c1002a0 0x7fb22c102be0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.915+0000 7fb23276f640 1 -- 172.21.15.19:0/3350801801 >> 172.21.15.19:0/3350801801 conn(0x7fb22c0fbbd0 msgr2=0x7fb22c106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.915+0000 7fb23276f640 1 -- 172.21.15.19:0/3350801801 shutdown_connections 2024-04-25T12:07:10.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:10.915+0000 7fb23276f640 1 -- 172.21.15.19:0/3350801801 wait complete. 2024-04-25T12:07:10.926 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T12:07:11.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.042+0000 7fd304b8e640 1 -- 172.21.15.19:0/2591997626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd300102aa0 msgr2=0x7fd3001093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:11.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.042+0000 7fd304b8e640 1 --2- 172.21.15.19:0/2591997626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd300102aa0 0x7fd3001093c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009be0 tx=0x7fd2f802a6c0 comp rx=0 tx=0).stop 2024-04-25T12:07:11.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.042+0000 7fd304b8e640 1 -- 172.21.15.19:0/2591997626 shutdown_connections 2024-04-25T12:07:11.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.042+0000 7fd304b8e640 1 --2- 172.21.15.19:0/2591997626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd300102aa0 0x7fd3001093c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:11.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.042+0000 7fd304b8e640 1 --2- 172.21.15.19:0/2591997626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3001020e0 0x7fd300102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:11.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.042+0000 7fd304b8e640 1 --2- 172.21.15.19:0/2591997626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd300100f10 0x7fd300101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:11.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.042+0000 7fd304b8e640 1 -- 172.21.15.19:0/2591997626 >> 172.21.15.19:0/2591997626 conn(0x7fd3000fc4d0 msgr2=0x7fd3000fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:11.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.042+0000 7fd304b8e640 1 -- 172.21.15.19:0/2591997626 shutdown_connections 2024-04-25T12:07:11.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.042+0000 7fd304b8e640 1 -- 172.21.15.19:0/2591997626 wait complete. 2024-04-25T12:07:11.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd304b8e640 1 Processor -- start 2024-04-25T12:07:11.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd304b8e640 1 -- start start 2024-04-25T12:07:11.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd304b8e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd300100f10 0x7fd300104f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:11.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd304b8e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3001020e0 0x7fd300103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:11.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd304b8e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd300102aa0 0x7fd300103a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:11.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd304b8e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd30010cb60 con 0x7fd300102aa0 2024-04-25T12:07:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd304b8e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd30010c9e0 con 0x7fd3001020e0 2024-04-25T12:07:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd304b8e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fd30010cce0 con 0x7fd300100f10 2024-04-25T12:07:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd2fed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd300102aa0 0x7fd300103a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd2fdd9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3001020e0 0x7fd300103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd2fed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd300102aa0 0x7fd300103a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56518/0 (socket says 172.21.15.19:56518) 2024-04-25T12:07:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd2fe59c640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd300100f10 0x7fd300104f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd2fed9d640 1 -- 172.21.15.19:0/2267045380 learned_addr learned my addr 172.21.15.19:0/2267045380 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:07:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd2fed9d640 1 -- 172.21.15.19:0/2267045380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd300100f10 msgr2=0x7fd300104f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.043+0000 7fd2fed9d640 1 --2- 172.21.15.19:0/2267045380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd300100f10 0x7fd300104f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.044+0000 7fd2fed9d640 1 -- 172.21.15.19:0/2267045380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3001020e0 msgr2=0x7fd300103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.044+0000 7fd2fed9d640 1 --2- 172.21.15.19:0/2267045380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3001020e0 0x7fd300103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.044+0000 7fd2fed9d640 1 -- 172.21.15.19:0/2267045380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd300104320 con 0x7fd300102aa0 2024-04-25T12:07:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.044+0000 7fd2fed9d640 1 --2- 172.21.15.19:0/2267045380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd300102aa0 0x7fd300103a90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8034040 tx=0x7fd2f8032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:11.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.044+0000 7fd2f77fe640 1 -- 172.21.15.19:0/2267045380 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8041020 con 0x7fd300102aa0 2024-04-25T12:07:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.044+0000 7fd2f77fe640 1 -- 172.21.15.19:0/2267045380 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fd2f8031910 con 0x7fd300102aa0 2024-04-25T12:07:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.044+0000 7fd304b8e640 1 -- 172.21.15.19:0/2267045380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3001a22f0 con 0x7fd300102aa0 2024-04-25T12:07:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.044+0000 7fd2f77fe640 1 -- 172.21.15.19:0/2267045380 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8036040 con 0x7fd300102aa0 2024-04-25T12:07:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.044+0000 7fd304b8e640 1 -- 172.21.15.19:0/2267045380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3001a2840 con 0x7fd300102aa0 2024-04-25T12:07:11.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.045+0000 7fd304b8e640 1 -- 172.21.15.19:0/2267045380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3001a2aa0 con 0x7fd300102aa0 2024-04-25T12:07:11.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.046+0000 7fd2f77fe640 1 -- 172.21.15.19:0/2267045380 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fd2f8044020 con 0x7fd300102aa0 2024-04-25T12:07:11.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.047+0000 7fd2f77fe640 1 --2- 172.21.15.19:0/2267045380 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fd2ec0a3a60 0x7fd2ec0a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:07:11.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.047+0000 7fd2f77fe640 1 -- 172.21.15.19:0/2267045380 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fd2f80f0d60 con 0x7fd300102aa0 2024-04-25T12:07:11.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.047+0000 7fd2fe59c640 1 --2- 172.21.15.19:0/2267045380 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fd2ec0a3a60 0x7fd2ec0a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:07:11.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.047+0000 7fd2fe59c640 1 --2- 172.21.15.19:0/2267045380 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fd2ec0a3a60 0x7fd2ec0a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd300100c30 tx=0x7fd2e8009260 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:07:11.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:10 smithi019 ceph-mon[29213]: [25/Apr/2024:12:07:09] ENGINE Bus STARTING 2024-04-25T12:07:11.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:10 smithi019 ceph-mon[29213]: [25/Apr/2024:12:07:09] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T12:07:11.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:10 smithi019 ceph-mon[29213]: mgrmap e22: x(active, since 1.01761s), standbys: y, z 2024-04-25T12:07:11.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.052+0000 7fd2f77fe640 1 -- 172.21.15.19:0/2267045380 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd2f804b020 con 0x7fd300102aa0 2024-04-25T12:07:11.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:10 smithi169 ceph-mon[32710]: [25/Apr/2024:12:07:09] ENGINE Bus STARTING 2024-04-25T12:07:11.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:10 smithi169 ceph-mon[32710]: [25/Apr/2024:12:07:09] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T12:07:11.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:10 smithi169 ceph-mon[32710]: mgrmap e22: x(active, since 1.01761s), standbys: y, z 2024-04-25T12:07:11.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:10 smithi150 ceph-mon[32250]: [25/Apr/2024:12:07:09] ENGINE Bus STARTING 2024-04-25T12:07:11.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:10 smithi150 ceph-mon[32250]: [25/Apr/2024:12:07:09] ENGINE Serving on https://172.21.15.19:7150 2024-04-25T12:07:11.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:10 smithi150 ceph-mon[32250]: mgrmap e22: x(active, since 1.01761s), standbys: y, z 2024-04-25T12:07:11.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.288+0000 7fd304b8e640 1 -- 172.21.15.19:0/2267045380 --> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd3000fe580 con 0x7fd2ec0a3a60 2024-04-25T12:07:11.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.296+0000 7fd2f77fe640 1 -- 172.21.15.19:0/2267045380 <== mgr.15381 v2:172.21.15.19:6800/3657854393 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd3000fe580 con 0x7fd2ec0a3a60 2024-04-25T12:07:11.294 INFO:teuthology.orchestra.run.smithi019.stdout:/ 1m 6m3h 2024-04-25T12:07:11.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.298+0000 7fd304b8e640 1 -- 172.21.15.19:0/2267045380 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fd2ec0a3a60 msgr2=0x7fd2ec0a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:11.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.298+0000 7fd304b8e640 1 --2- 172.21.15.19:0/2267045380 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fd2ec0a3a60 0x7fd2ec0a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd300100c30 tx=0x7fd2e8009260 comp rx=0 tx=0).stop 2024-04-25T12:07:11.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.298+0000 7fd304b8e640 1 -- 172.21.15.19:0/2267045380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd300102aa0 msgr2=0x7fd300103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:07:11.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.298+0000 7fd304b8e640 1 --2- 172.21.15.19:0/2267045380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd300102aa0 0x7fd300103a90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8034040 tx=0x7fd2f8032f90 comp rx=0 tx=0).stop 2024-04-25T12:07:11.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.298+0000 7fd304b8e640 1 -- 172.21.15.19:0/2267045380 shutdown_connections 2024-04-25T12:07:11.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.298+0000 7fd304b8e640 1 --2- 172.21.15.19:0/2267045380 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fd2ec0a3a60 0x7fd2ec0a5f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:11.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.298+0000 7fd304b8e640 1 --2- 172.21.15.19:0/2267045380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd300102aa0 0x7fd300103a90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:11.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.298+0000 7fd304b8e640 1 --2- 172.21.15.19:0/2267045380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3001020e0 0x7fd300103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:11.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.298+0000 7fd304b8e640 1 --2- 172.21.15.19:0/2267045380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd300100f10 0x7fd300104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:07:11.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.298+0000 7fd304b8e640 1 -- 172.21.15.19:0/2267045380 >> 172.21.15.19:0/2267045380 conn(0x7fd3000fc4d0 msgr2=0x7fd3000fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:07:11.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.298+0000 7fd304b8e640 1 -- 172.21.15.19:0/2267045380 shutdown_connections 2024-04-25T12:07:11.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:07:11.298+0000 7fd304b8e640 1 -- 172.21.15.19:0/2267045380 wait complete. 2024-04-25T12:07:11.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T12:07:11.337 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T12:07:11.349 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T12:07:11.393 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T12:07:11.393 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T12:07:11.396 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T12:07:11.428 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa/tasks/quiescer.py", line 386, in task cluster_name = ctx.config['cluster'] KeyError: 'cluster' 2024-04-25T12:07:11.665 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=6087444a5d214a5aa3392b47e5b0fa5c Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa/tasks/quiescer.py", line 386, in task cluster_name = ctx.config['cluster'] KeyError: 'cluster' 2024-04-25T12:07:11.667 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T12:07:11.677 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T12:07:11.709 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T12:07:11.723 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T12:07:11.743 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T12:07:11.743 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T12:07:11.743 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T12:07:11.744 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T12:07:11.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T12:07:12.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:11 smithi019 ceph-mon[29213]: [25/Apr/2024:12:07:09] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T12:07:12.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:11 smithi019 ceph-mon[29213]: [25/Apr/2024:12:07:09] ENGINE Bus STARTED 2024-04-25T12:07:12.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:11 smithi019 ceph-mon[29213]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:07:12.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:11 smithi019 ceph-mon[29213]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:07:12.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:11 smithi019 ceph-mon[29213]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:07:12.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:11 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:11 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:11 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:11 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:11 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:11 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:11 smithi169 ceph-mon[32710]: [25/Apr/2024:12:07:09] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T12:07:12.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:11 smithi169 ceph-mon[32710]: [25/Apr/2024:12:07:09] ENGINE Bus STARTED 2024-04-25T12:07:12.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:11 smithi169 ceph-mon[32710]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:07:12.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:11 smithi169 ceph-mon[32710]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:07:12.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:11 smithi169 ceph-mon[32710]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:07:12.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:11 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:11 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:11 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:11 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:11 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:11 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:11 smithi150 ceph-mon[32250]: [25/Apr/2024:12:07:09] ENGINE Serving on http://172.21.15.19:8765 2024-04-25T12:07:12.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:11 smithi150 ceph-mon[32250]: [25/Apr/2024:12:07:09] ENGINE Bus STARTED 2024-04-25T12:07:12.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:11 smithi150 ceph-mon[32250]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:07:12.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:11 smithi150 ceph-mon[32250]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:07:12.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:11 smithi150 ceph-mon[32250]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:07:12.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:11 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:11 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:11 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:11 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:11 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:12.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:11 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:07:13.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:07:13.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:07:13.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T12:07:13.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:07:13.160 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:12 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:07:13.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:07:13.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:07:13.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:07:13.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T12:07:13.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:07:13.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:12 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:07:13.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:07:13.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:07:13.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:07:13.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T12:07:13.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:13.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:07:13.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:07:13.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:12 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:07:14.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T12:07:14.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: Unable to set osd_memory_target on smithi169 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T12:07:14.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-25T12:07:14.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: Unable to set osd_memory_target on smithi150 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T12:07:14.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-25T12:07:14.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T12:07:14.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T12:07:14.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T12:07:14.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T12:07:14.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: Updating smithi150:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:07:14.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:07:14.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:07:14.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:07:14.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:13 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T12:07:14.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: Unable to set osd_memory_target on smithi169 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T12:07:14.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-25T12:07:14.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: Unable to set osd_memory_target on smithi150 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T12:07:14.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-25T12:07:14.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T12:07:14.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T12:07:14.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T12:07:14.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T12:07:14.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: Updating smithi150:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:07:14.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:07:14.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:07:14.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:07:14.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:13 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T12:07:14.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: Unable to set osd_memory_target on smithi169 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T12:07:14.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-25T12:07:14.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: Unable to set osd_memory_target on smithi150 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T12:07:14.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-25T12:07:14.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T12:07:14.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: Updating smithi019:/etc/ceph/ceph.conf 2024-04-25T12:07:14.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-25T12:07:14.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: Updating smithi169:/etc/ceph/ceph.conf 2024-04-25T12:07:14.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: Updating smithi150:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:07:14.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:14.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:07:14.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:07:14.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:07:14.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:13 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:07:15.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:14 smithi169 ceph-mon[32710]: Updating smithi019:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:07:15.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:14 smithi169 ceph-mon[32710]: Updating smithi169:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:07:15.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:14 smithi169 ceph-mon[32710]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:07:15.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:14 smithi169 ceph-mon[32710]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:07:15.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:14 smithi169 ceph-mon[32710]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:07:15.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:14 smithi169 ceph-mon[32710]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:07:15.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:14 smithi169 ceph-mon[32710]: Updating smithi150:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:07:15.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:14 smithi169 ceph-mon[32710]: Updating smithi019:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:07:15.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:14 smithi169 ceph-mon[32710]: Updating smithi169:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:07:15.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:14 smithi169 ceph-mon[32710]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T12:07:15.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:14 smithi150 ceph-mon[32250]: Updating smithi019:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:07:15.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:14 smithi150 ceph-mon[32250]: Updating smithi169:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:07:15.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:14 smithi150 ceph-mon[32250]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:07:15.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:14 smithi150 ceph-mon[32250]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:07:15.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:14 smithi150 ceph-mon[32250]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:07:15.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:14 smithi150 ceph-mon[32250]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:07:15.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:14 smithi150 ceph-mon[32250]: Updating smithi150:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:07:15.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:14 smithi150 ceph-mon[32250]: Updating smithi019:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:07:15.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:14 smithi150 ceph-mon[32250]: Updating smithi169:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:07:15.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:14 smithi150 ceph-mon[32250]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T12:07:15.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:14 smithi019 ceph-mon[29213]: Updating smithi019:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:07:15.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:14 smithi019 ceph-mon[29213]: Updating smithi169:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T12:07:15.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:14 smithi019 ceph-mon[29213]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:07:15.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:14 smithi019 ceph-mon[29213]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:07:15.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:14 smithi019 ceph-mon[29213]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-04-25T12:07:15.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:14 smithi019 ceph-mon[29213]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:07:15.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:14 smithi019 ceph-mon[29213]: Updating smithi150:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:07:15.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:14 smithi019 ceph-mon[29213]: Updating smithi019:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:07:15.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:14 smithi019 ceph-mon[29213]: Updating smithi169:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T12:07:15.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:14 smithi019 ceph-mon[29213]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T12:07:16.156 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:15 smithi169 ceph-mon[32710]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:07:16.163 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:15 smithi150 ceph-mon[32250]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:07:16.199 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:15 smithi019 ceph-mon[29213]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T12:07:18.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:18 smithi169 ceph-mon[32710]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 131 KiB/s wr, 77 op/s 2024-04-25T12:07:18.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:18 smithi150 ceph-mon[32250]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 131 KiB/s wr, 77 op/s 2024-04-25T12:07:18.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:18 smithi019 ceph-mon[29213]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 131 KiB/s wr, 77 op/s 2024-04-25T12:07:20.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:20 smithi169 ceph-mon[32710]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 60 op/s 2024-04-25T12:07:20.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:20 smithi150 ceph-mon[32250]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 60 op/s 2024-04-25T12:07:20.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:20 smithi019 ceph-mon[29213]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 60 op/s 2024-04-25T12:07:22.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:22 smithi169 ceph-mon[32710]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-04-25T12:07:22.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:22 smithi150 ceph-mon[32250]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-04-25T12:07:22.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:22 smithi019 ceph-mon[29213]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-04-25T12:07:24.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:24 smithi169 ceph-mon[32710]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T12:07:24.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:24 smithi150 ceph-mon[32250]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T12:07:24.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:24 smithi019 ceph-mon[29213]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T12:07:26.406 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:26 smithi169 ceph-mon[32710]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T12:07:26.413 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:26 smithi150 ceph-mon[32250]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T12:07:26.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:26 smithi019 ceph-mon[29213]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T12:07:28.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:28 smithi150 ceph-mon[32250]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 53 op/s 2024-04-25T12:07:28.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:28 smithi019 ceph-mon[29213]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 53 op/s 2024-04-25T12:07:28.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:28 smithi169 ceph-mon[32710]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 53 op/s 2024-04-25T12:07:30.414 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:30 smithi150 ceph-mon[32250]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:30.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:30 smithi019 ceph-mon[29213]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:30.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:30 smithi169 ceph-mon[32710]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:32.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:32 smithi019 ceph-mon[29213]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:32.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:32 smithi169 ceph-mon[32710]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:32.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:32 smithi150 ceph-mon[32250]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:34.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:34 smithi019 ceph-mon[29213]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:34.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:34 smithi169 ceph-mon[32710]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:34.663 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:34 smithi150 ceph-mon[32250]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:36.459 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:36 smithi169 ceph-mon[32710]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T12:07:36.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:36 smithi019 ceph-mon[29213]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T12:07:36.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:36 smithi150 ceph-mon[32250]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-25T12:07:38.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:38 smithi019 ceph-mon[29213]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:38.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:38 smithi169 ceph-mon[32710]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:38.663 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:38 smithi150 ceph-mon[32250]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:40.466 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:40 smithi150 ceph-mon[32250]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:40.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:40 smithi019 ceph-mon[29213]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:40.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:40 smithi169 ceph-mon[32710]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:42.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:42 smithi019 ceph-mon[29213]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:42.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:42 smithi169 ceph-mon[32710]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:42.663 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:42 smithi150 ceph-mon[32250]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:44.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:44 smithi019 ceph-mon[29213]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:44.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:44 smithi169 ceph-mon[32710]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:44.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:44 smithi150 ceph-mon[32250]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:46.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:46 smithi019 ceph-mon[29213]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T12:07:46.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:46 smithi169 ceph-mon[32710]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T12:07:46.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:46 smithi150 ceph-mon[32250]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T12:07:48.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:48 smithi019 ceph-mon[29213]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T12:07:48.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:48 smithi169 ceph-mon[32710]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T12:07:48.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:48 smithi150 ceph-mon[32250]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-25T12:07:50.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:50 smithi019 ceph-mon[29213]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:50.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:50 smithi169 ceph-mon[32710]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:50.663 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:50 smithi150 ceph-mon[32250]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:52.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:52 smithi019 ceph-mon[29213]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:52.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:52 smithi169 ceph-mon[32710]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:52.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:52 smithi150 ceph-mon[32250]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:54.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:54 smithi019 ceph-mon[29213]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:54.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:54 smithi169 ceph-mon[32710]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:54.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:54 smithi150 ceph-mon[32250]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:07:56.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:56 smithi019 ceph-mon[29213]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:56.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:56 smithi169 ceph-mon[32710]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:56.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:56 smithi150 ceph-mon[32250]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:07:58.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:07:58 smithi019 ceph-mon[29213]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T12:07:58.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:07:58 smithi169 ceph-mon[32710]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T12:07:58.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:07:58 smithi150 ceph-mon[32250]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T12:08:00.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:00 smithi019 ceph-mon[29213]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:00.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:00 smithi169 ceph-mon[32710]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:00.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:00 smithi150 ceph-mon[32250]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:01.048 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:08:00 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:08:00.573+0000 7f24ac6d3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:08:01.049 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:08:00 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:08:00.573+0000 7f24ac6d3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:08:01.049 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:08:00 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:08:00.573+0000 7f24ac6d3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:08:01.049 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:08:00 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:08:00.573+0000 7f24ac6d3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:08:01.049 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:08:00 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:08:00.573+0000 7f24ac6d3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T12:08:02.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:02 smithi019 ceph-mon[29213]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:08:02.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:02 smithi019 ceph-mon[29213]: mgrmap e25: x(active, since 52s), standbys: y, z 2024-04-25T12:08:02.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:02 smithi169 ceph-mon[32710]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:08:02.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:02 smithi169 ceph-mon[32710]: mgrmap e25: x(active, since 52s), standbys: y, z 2024-04-25T12:08:02.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:02 smithi150 ceph-mon[32250]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:08:02.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:02 smithi150 ceph-mon[32250]: mgrmap e25: x(active, since 52s), standbys: y, z 2024-04-25T12:08:04.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:04 smithi019 ceph-mon[29213]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:04.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:04 smithi169 ceph-mon[32710]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:04.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:04 smithi150 ceph-mon[32250]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:06.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:06 smithi019 ceph-mon[29213]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:08:06.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:06 smithi169 ceph-mon[32710]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:08:06.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:06 smithi150 ceph-mon[32250]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:08:08.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:08 smithi019 ceph-mon[29213]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:08:08.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:08 smithi169 ceph-mon[32710]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:08:08.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:08 smithi150 ceph-mon[32250]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:08:10.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:10 smithi019 ceph-mon[29213]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:08:10.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:10 smithi169 ceph-mon[32710]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:08:10.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:10 smithi150 ceph-mon[32250]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:08:12.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:12 smithi019 ceph-mon[29213]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-25T12:08:12.656 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:12 smithi169 ceph-mon[32710]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-25T12:08:12.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:12 smithi150 ceph-mon[32250]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-25T12:08:14.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:14 smithi019 ceph-mon[29213]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:08:14.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:14 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:08:14.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:14 smithi169 ceph-mon[32710]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:08:14.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:14 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:08:14.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:14 smithi150 ceph-mon[32250]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:08:14.664 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:14 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:08:15.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:15 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:08:15.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:15 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:08:15.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:15 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:08:15.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:15 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:08:15.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:15 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:08:15.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:15 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:08:15.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:15 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:08:15.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:15 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:08:15.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:15 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:08:15.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:15 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:08:15.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:15 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:08:15.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:15 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:08:15.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:15 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:08:15.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:15 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:08:15.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:15 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:08:16.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:16 smithi019 ceph-mon[29213]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-25T12:08:16.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:16 smithi169 ceph-mon[32710]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-25T12:08:16.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:16 smithi150 ceph-mon[32250]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-25T12:08:18.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:18 smithi019 ceph-mon[29213]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:08:18.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:18 smithi169 ceph-mon[32710]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:08:18.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:18 smithi150 ceph-mon[32250]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:08:20.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:20 smithi019 ceph-mon[29213]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:08:20.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:20 smithi169 ceph-mon[32710]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:08:20.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:20 smithi150 ceph-mon[32250]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:08:22.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:22 smithi019 ceph-mon[29213]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:08:22.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:22 smithi169 ceph-mon[32710]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:08:22.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:22 smithi150 ceph-mon[32250]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:08:24.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:24 smithi019 ceph-mon[29213]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:24.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:24 smithi169 ceph-mon[32710]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:24.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:24 smithi150 ceph-mon[32250]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:26.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:26 smithi019 ceph-mon[29213]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:08:26.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:26 smithi169 ceph-mon[32710]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:08:26.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:26 smithi150 ceph-mon[32250]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:08:28.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:28 smithi019 ceph-mon[29213]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:28.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:28 smithi169 ceph-mon[32710]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:28.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:28 smithi150 ceph-mon[32250]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:30.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:30 smithi019 ceph-mon[29213]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:30.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:30 smithi169 ceph-mon[32710]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:30.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:30 smithi150 ceph-mon[32250]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:32.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:32 smithi019 ceph-mon[29213]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:08:32.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:32 smithi169 ceph-mon[32710]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:08:32.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:32 smithi150 ceph-mon[32250]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:08:34.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:34 smithi019 ceph-mon[29213]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:34.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:34 smithi169 ceph-mon[32710]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:34.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:34 smithi150 ceph-mon[32250]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:36.798 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:36 smithi019 ceph-mon[29213]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:08:36.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:36 smithi169 ceph-mon[32710]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:08:36.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:36 smithi150 ceph-mon[32250]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:08:38.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:38 smithi019 ceph-mon[29213]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:38.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:38 smithi169 ceph-mon[32710]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:38.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:38 smithi150 ceph-mon[32250]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:40.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:40 smithi019 ceph-mon[29213]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:40.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:40 smithi169 ceph-mon[32710]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:40.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:40 smithi150 ceph-mon[32250]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:42.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:42 smithi019 ceph-mon[29213]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T12:08:42.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:42 smithi019 ceph-mon[29213]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T12:08:42.906 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:42 smithi169 ceph-mon[32710]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T12:08:42.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:42 smithi169 ceph-mon[32710]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T12:08:42.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:42 smithi150 ceph-mon[32250]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T12:08:42.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:42 smithi150 ceph-mon[32250]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T12:08:44.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:44 smithi169 ceph-mon[32710]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T12:08:44.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:44 smithi150 ceph-mon[32250]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T12:08:45.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:44 smithi019 ceph-mon[29213]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T12:08:46.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:46 smithi019 ceph-mon[29213]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T12:08:46.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:46 smithi169 ceph-mon[32710]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T12:08:46.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:46 smithi150 ceph-mon[32250]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T12:08:48.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:48 smithi169 ceph-mon[32710]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T12:08:48.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:48 smithi150 ceph-mon[32250]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T12:08:49.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:48 smithi019 ceph-mon[29213]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T12:08:50.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:50 smithi169 ceph-mon[32710]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T12:08:50.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:50 smithi150 ceph-mon[32250]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T12:08:51.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:50 smithi019 ceph-mon[29213]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-25T12:08:52.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:52 smithi169 ceph-mon[32710]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T12:08:52.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:52 smithi150 ceph-mon[32250]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T12:08:53.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:52 smithi019 ceph-mon[29213]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-25T12:08:54.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:54 smithi169 ceph-mon[32710]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:54.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:54 smithi150 ceph-mon[32250]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:55.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:54 smithi019 ceph-mon[29213]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:56.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:56 smithi169 ceph-mon[32710]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-04-25T12:08:56.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:56 smithi150 ceph-mon[32250]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-04-25T12:08:57.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:56 smithi019 ceph-mon[29213]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-04-25T12:08:58.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:08:58 smithi169 ceph-mon[32710]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:58.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:08:58 smithi150 ceph-mon[32250]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:08:59.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:08:58 smithi019 ceph-mon[29213]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:00.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:00 smithi169 ceph-mon[32710]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:00.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:00 smithi150 ceph-mon[32250]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:01.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:00 smithi019 ceph-mon[29213]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:02.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:02 smithi169 ceph-mon[32710]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:02.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:02 smithi150 ceph-mon[32250]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:03.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:02 smithi019 ceph-mon[29213]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:04.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:04 smithi169 ceph-mon[32710]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:04.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:04 smithi150 ceph-mon[32250]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:05.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:04 smithi019 ceph-mon[29213]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:06.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:06 smithi169 ceph-mon[32710]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:09:06.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:06 smithi150 ceph-mon[32250]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:09:07.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:06 smithi019 ceph-mon[29213]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:09:08.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:08 smithi169 ceph-mon[32710]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:09:08.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:08 smithi150 ceph-mon[32250]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:09:09.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:08 smithi019 ceph-mon[29213]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:09:10.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:10 smithi169 ceph-mon[32710]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:09:10.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:10 smithi150 ceph-mon[32250]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:09:11.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:10 smithi019 ceph-mon[29213]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:09:12.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:12 smithi169 ceph-mon[32710]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:09:12.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:12 smithi150 ceph-mon[32250]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:09:13.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:12 smithi019 ceph-mon[29213]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:09:14.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:14 smithi169 ceph-mon[32710]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:09:14.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:14 smithi150 ceph-mon[32250]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:09:15.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:14 smithi019 ceph-mon[29213]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:09:15.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:15 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:09:15.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:15 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:09:16.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:15 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:09:16.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:16 smithi169 ceph-mon[32710]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:09:16.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:16 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:16.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:16 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:16.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:16 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:16.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:16 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:16.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:16 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:09:16.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:16 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:09:16.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:16 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:16.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:16 smithi150 ceph-mon[32250]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:09:16.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:16 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:16.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:16 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:16.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:16 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:16.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:16 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:16.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:16 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:09:16.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:16 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:09:16.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:16 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:16.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:16 smithi019 ceph-mon[29213]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:09:16.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:16 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:16.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:16 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:16.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:16 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:16.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:16 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:16.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:16 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:09:16.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:16 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:09:16.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:16 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:09:18.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:18 smithi169 ceph-mon[32710]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:09:18.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:18 smithi150 ceph-mon[32250]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:09:19.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:18 smithi019 ceph-mon[29213]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:09:20.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:20 smithi169 ceph-mon[32710]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:09:20.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:20 smithi150 ceph-mon[32250]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:09:21.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:20 smithi019 ceph-mon[29213]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:09:22.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:22 smithi169 ceph-mon[32710]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:09:22.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:22 smithi150 ceph-mon[32250]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:09:23.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:22 smithi019 ceph-mon[29213]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:09:24.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:24 smithi169 ceph-mon[32710]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:24.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:24 smithi150 ceph-mon[32250]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:25.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:24 smithi019 ceph-mon[29213]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:26.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:26 smithi169 ceph-mon[32710]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:26.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:26 smithi150 ceph-mon[32250]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:27.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:26 smithi019 ceph-mon[29213]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:28.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:28 smithi169 ceph-mon[32710]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:28.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:28 smithi150 ceph-mon[32250]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:29.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:28 smithi019 ceph-mon[29213]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:30.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:30 smithi169 ceph-mon[32710]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:30.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:30 smithi150 ceph-mon[32250]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:31.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:30 smithi019 ceph-mon[29213]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:32.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:32 smithi169 ceph-mon[32710]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:32.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:32 smithi150 ceph-mon[32250]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:33.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:32 smithi019 ceph-mon[29213]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:34.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:34 smithi169 ceph-mon[32710]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:34.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:34 smithi150 ceph-mon[32250]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:35.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:34 smithi019 ceph-mon[29213]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:36.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:36 smithi169 ceph-mon[32710]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:36.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:36 smithi150 ceph-mon[32250]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:37.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:36 smithi019 ceph-mon[29213]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:38.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:38 smithi169 ceph-mon[32710]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:38.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:38 smithi150 ceph-mon[32250]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:39.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:38 smithi019 ceph-mon[29213]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:40.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:40 smithi169 ceph-mon[32710]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:40.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:40 smithi150 ceph-mon[32250]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:41.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:40 smithi019 ceph-mon[29213]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:42.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:42 smithi169 ceph-mon[32710]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:42.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:42 smithi150 ceph-mon[32250]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:43.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:42 smithi019 ceph-mon[29213]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:44.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:44 smithi169 ceph-mon[32710]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:44.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:44 smithi150 ceph-mon[32250]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:45.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:44 smithi019 ceph-mon[29213]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:46.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:46 smithi169 ceph-mon[32710]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:46.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:46 smithi150 ceph-mon[32250]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:47.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:46 smithi019 ceph-mon[29213]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:48.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:48 smithi169 ceph-mon[32710]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:48.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:48 smithi150 ceph-mon[32250]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:49.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:48 smithi019 ceph-mon[29213]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:50.907 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:50 smithi169 ceph-mon[32710]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:50.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:50 smithi150 ceph-mon[32250]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:51.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:50 smithi019 ceph-mon[29213]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:52.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:52 smithi150 ceph-mon[32250]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:53.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:52 smithi019 ceph-mon[29213]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:53.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:52 smithi169 ceph-mon[32710]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:54.914 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:54 smithi150 ceph-mon[32250]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:55.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:54 smithi019 ceph-mon[29213]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:55.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:54 smithi169 ceph-mon[32710]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:57.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:56 smithi019 ceph-mon[29213]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:57.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:56 smithi169 ceph-mon[32710]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:57.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:56 smithi150 ceph-mon[32250]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:09:59.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:09:58 smithi019 ceph-mon[29213]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:59.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:09:58 smithi169 ceph-mon[32710]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:09:59.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:09:58 smithi150 ceph-mon[32250]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:01.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:00 smithi019 ceph-mon[29213]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:01.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:00 smithi019 ceph-mon[29213]: overall HEALTH_OK 2024-04-25T12:10:01.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:00 smithi169 ceph-mon[32710]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:01.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:00 smithi169 ceph-mon[32710]: overall HEALTH_OK 2024-04-25T12:10:01.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:00 smithi150 ceph-mon[32250]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:01.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:00 smithi150 ceph-mon[32250]: overall HEALTH_OK 2024-04-25T12:10:03.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:02 smithi019 ceph-mon[29213]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:03.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:02 smithi169 ceph-mon[32710]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:03.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:02 smithi150 ceph-mon[32250]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:05.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:04 smithi019 ceph-mon[29213]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:05.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:04 smithi169 ceph-mon[32710]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:05.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:04 smithi150 ceph-mon[32250]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:07.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:06 smithi019 ceph-mon[29213]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:10:07.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:06 smithi169 ceph-mon[32710]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:10:07.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:06 smithi150 ceph-mon[32250]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:10:09.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:08 smithi019 ceph-mon[29213]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:10:09.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:08 smithi169 ceph-mon[32710]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:10:09.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:08 smithi150 ceph-mon[32250]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:10:10.967 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:10 smithi150 ceph-mon[32250]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:10:11.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:10 smithi019 ceph-mon[29213]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:10:11.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:10 smithi169 ceph-mon[32710]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:10:13.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:12 smithi019 ceph-mon[29213]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:10:13.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:12 smithi169 ceph-mon[32710]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:10:13.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:12 smithi150 ceph-mon[32250]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:10:15.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:14 smithi019 ceph-mon[29213]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:10:15.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:14 smithi169 ceph-mon[32710]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:10:15.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:14 smithi150 ceph-mon[32250]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:10:17.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:16 smithi019 ceph-mon[29213]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:10:17.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:16 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:10:17.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:16 smithi169 ceph-mon[32710]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:10:17.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:16 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:10:17.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:16 smithi150 ceph-mon[32250]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:10:17.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:16 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:10:18.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:17 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:10:18.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:17 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:10:18.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:17 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:10:18.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:17 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:10:18.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:17 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:10:18.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:17 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:10:18.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:17 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:10:18.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:17 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:10:18.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:17 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:10:19.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:18 smithi019 ceph-mon[29213]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:10:19.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:18 smithi169 ceph-mon[32710]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:10:19.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:18 smithi150 ceph-mon[32250]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:10:21.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:20 smithi019 ceph-mon[29213]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:10:21.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:20 smithi169 ceph-mon[32710]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:10:21.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:20 smithi150 ceph-mon[32250]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:10:23.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:22 smithi019 ceph-mon[29213]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:10:23.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:22 smithi169 ceph-mon[32710]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:10:23.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:22 smithi150 ceph-mon[32250]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:10:25.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:24 smithi019 ceph-mon[29213]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:25.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:24 smithi169 ceph-mon[32710]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:25.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:24 smithi150 ceph-mon[32250]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:27.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:26 smithi019 ceph-mon[29213]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:27.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:26 smithi169 ceph-mon[32710]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:27.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:26 smithi150 ceph-mon[32250]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:29.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:28 smithi019 ceph-mon[29213]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:29.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:28 smithi169 ceph-mon[32710]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:29.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:28 smithi150 ceph-mon[32250]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:31.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:30 smithi019 ceph-mon[29213]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:31.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:30 smithi169 ceph-mon[32710]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:31.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:30 smithi150 ceph-mon[32250]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:33.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:32 smithi019 ceph-mon[29213]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:33.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:32 smithi169 ceph-mon[32710]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:33.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:32 smithi150 ceph-mon[32250]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:35.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:34 smithi019 ceph-mon[29213]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:35.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:34 smithi169 ceph-mon[32710]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:35.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:34 smithi150 ceph-mon[32250]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:37.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:36 smithi019 ceph-mon[29213]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:37.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:36 smithi169 ceph-mon[32710]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:37.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:36 smithi150 ceph-mon[32250]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:39.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:38 smithi019 ceph-mon[29213]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:39.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:38 smithi169 ceph-mon[32710]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:39.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:38 smithi150 ceph-mon[32250]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:41.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:40 smithi019 ceph-mon[29213]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:41.164 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:40 smithi169 ceph-mon[32710]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:41.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:40 smithi150 ceph-mon[32250]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:43.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:42 smithi019 ceph-mon[29213]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:43.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:42 smithi169 ceph-mon[32710]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:43.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:42 smithi150 ceph-mon[32250]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:45.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:44 smithi019 ceph-mon[29213]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:45.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:44 smithi169 ceph-mon[32710]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:45.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:44 smithi150 ceph-mon[32250]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:47.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:46 smithi019 ceph-mon[29213]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:47.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:46 smithi169 ceph-mon[32710]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:47.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:46 smithi150 ceph-mon[32250]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:49.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:48 smithi019 ceph-mon[29213]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:49.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:48 smithi169 ceph-mon[32710]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:49.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:48 smithi150 ceph-mon[32250]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:51.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:50 smithi019 ceph-mon[29213]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:51.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:50 smithi169 ceph-mon[32710]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:51.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:50 smithi150 ceph-mon[32250]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:53.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:52 smithi019 ceph-mon[29213]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:53.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:52 smithi169 ceph-mon[32710]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:53.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:52 smithi150 ceph-mon[32250]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:55.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:54 smithi019 ceph-mon[29213]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:55.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:54 smithi169 ceph-mon[32710]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:55.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:54 smithi150 ceph-mon[32250]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:57.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:56 smithi019 ceph-mon[29213]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:57.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:56 smithi169 ceph-mon[32710]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:57.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:56 smithi150 ceph-mon[32250]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:10:59.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:10:58 smithi019 ceph-mon[29213]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:59.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:10:58 smithi169 ceph-mon[32710]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:10:59.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:10:58 smithi150 ceph-mon[32250]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:01.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:00 smithi019 ceph-mon[29213]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:01.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:00 smithi169 ceph-mon[32710]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:01.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:00 smithi150 ceph-mon[32250]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:03.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:02 smithi019 ceph-mon[29213]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:03.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:02 smithi169 ceph-mon[32710]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:03.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:02 smithi150 ceph-mon[32250]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:05.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:04 smithi019 ceph-mon[29213]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:05.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:04 smithi169 ceph-mon[32710]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:05.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:04 smithi150 ceph-mon[32250]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:07.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:06 smithi019 ceph-mon[29213]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:11:07.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:06 smithi169 ceph-mon[32710]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:11:07.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:06 smithi150 ceph-mon[32250]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:11:09.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:08 smithi019 ceph-mon[29213]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:11:09.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:08 smithi169 ceph-mon[32710]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:11:09.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:08 smithi150 ceph-mon[32250]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:11:11.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:10 smithi019 ceph-mon[29213]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:11:11.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:10 smithi169 ceph-mon[32710]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:11:11.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:10 smithi150 ceph-mon[32250]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:11:13.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:12 smithi019 ceph-mon[29213]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:11:13.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:12 smithi169 ceph-mon[32710]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:11:13.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:12 smithi150 ceph-mon[32250]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:11:15.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:14 smithi019 ceph-mon[29213]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:11:15.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:14 smithi169 ceph-mon[32710]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:11:15.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:14 smithi150 ceph-mon[32250]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:11:17.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:16 smithi019 ceph-mon[29213]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:11:17.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:16 smithi169 ceph-mon[32710]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:11:17.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:16 smithi150 ceph-mon[32250]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:11:18.049 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:17 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:11:18.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:17 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:11:18.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:17 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:11:19.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:18 smithi169 ceph-mon[32710]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:11:19.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:18 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:11:19.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:18 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:11:19.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:18 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:11:19.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:18 smithi150 ceph-mon[32250]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:11:19.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:18 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:11:19.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:18 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:11:19.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:18 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:11:19.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:18 smithi019 ceph-mon[29213]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:11:19.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:18 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:11:19.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:18 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:11:19.300 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:18 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:11:21.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:20 smithi169 ceph-mon[32710]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:11:21.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:20 smithi150 ceph-mon[32250]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:11:21.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:20 smithi019 ceph-mon[29213]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:11:23.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:22 smithi169 ceph-mon[32710]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:11:23.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:22 smithi150 ceph-mon[32250]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:11:23.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:22 smithi019 ceph-mon[29213]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:11:25.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:24 smithi169 ceph-mon[32710]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:25.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:24 smithi150 ceph-mon[32250]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:25.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:24 smithi019 ceph-mon[29213]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:27.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:26 smithi169 ceph-mon[32710]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:27.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:26 smithi150 ceph-mon[32250]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:27.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:26 smithi019 ceph-mon[29213]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:29.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:28 smithi169 ceph-mon[32710]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:29.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:28 smithi150 ceph-mon[32250]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:29.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:28 smithi019 ceph-mon[29213]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:31.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:30 smithi169 ceph-mon[32710]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:31.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:30 smithi150 ceph-mon[32250]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:31.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:30 smithi019 ceph-mon[29213]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:33.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:32 smithi169 ceph-mon[32710]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:33.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:32 smithi150 ceph-mon[32250]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:33.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:32 smithi019 ceph-mon[29213]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:35.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:34 smithi169 ceph-mon[32710]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:35.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:34 smithi150 ceph-mon[32250]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:35.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:34 smithi019 ceph-mon[29213]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:37.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:36 smithi169 ceph-mon[32710]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:37.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:36 smithi150 ceph-mon[32250]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:37.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:36 smithi019 ceph-mon[29213]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:39.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:38 smithi169 ceph-mon[32710]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:39.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:38 smithi150 ceph-mon[32250]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:39.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:38 smithi019 ceph-mon[29213]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:41.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:40 smithi169 ceph-mon[32710]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:41.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:40 smithi150 ceph-mon[32250]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:41.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:40 smithi019 ceph-mon[29213]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:43.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:42 smithi169 ceph-mon[32710]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:43.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:42 smithi150 ceph-mon[32250]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:43.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:42 smithi019 ceph-mon[29213]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:45.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:44 smithi169 ceph-mon[32710]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:45.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:44 smithi150 ceph-mon[32250]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:45.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:44 smithi019 ceph-mon[29213]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:47.102 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:46 smithi019 ceph-mon[29213]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:47.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:46 smithi169 ceph-mon[32710]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:47.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:46 smithi150 ceph-mon[32250]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:49.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:48 smithi169 ceph-mon[32710]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:49.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:48 smithi150 ceph-mon[32250]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:49.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:48 smithi019 ceph-mon[29213]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:51.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:50 smithi169 ceph-mon[32710]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:51.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:50 smithi150 ceph-mon[32250]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:51.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:50 smithi019 ceph-mon[29213]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:53.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:52 smithi169 ceph-mon[32710]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:53.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:52 smithi150 ceph-mon[32250]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:53.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:52 smithi019 ceph-mon[29213]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:55.157 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:54 smithi169 ceph-mon[32710]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:55.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:54 smithi150 ceph-mon[32250]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:55.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:54 smithi019 ceph-mon[29213]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:56.158 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:55 smithi169 ceph-mon[32710]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:56.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:55 smithi150 ceph-mon[32250]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:56.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:55 smithi019 ceph-mon[29213]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:11:58.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:11:58 smithi019 ceph-mon[29213]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:58.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:11:58 smithi169 ceph-mon[32710]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:11:58.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:11:58 smithi150 ceph-mon[32250]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:00.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:00 smithi019 ceph-mon[29213]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:00.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:00 smithi169 ceph-mon[32710]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:00.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:00 smithi150 ceph-mon[32250]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:02.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:02 smithi019 ceph-mon[29213]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:02.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:02 smithi169 ceph-mon[32710]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:02.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:02 smithi150 ceph-mon[32250]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:04.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:04 smithi019 ceph-mon[29213]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:04.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:04 smithi169 ceph-mon[32710]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:04.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:04 smithi150 ceph-mon[32250]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:06.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:06 smithi019 ceph-mon[29213]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:12:06.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:06 smithi169 ceph-mon[32710]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:12:06.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:06 smithi150 ceph-mon[32250]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:12:08.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:08 smithi019 ceph-mon[29213]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:12:08.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:08 smithi169 ceph-mon[32710]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:12:08.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:08 smithi150 ceph-mon[32250]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:12:10.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:10 smithi019 ceph-mon[29213]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:12:10.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:10 smithi169 ceph-mon[32710]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:12:10.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:10 smithi150 ceph-mon[32250]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:12:12.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:12 smithi019 ceph-mon[29213]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:12:12.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:12 smithi169 ceph-mon[32710]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:12:12.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:12 smithi150 ceph-mon[32250]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:12:14.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:14 smithi019 ceph-mon[29213]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:12:14.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:14 smithi169 ceph-mon[32710]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:12:14.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:14 smithi150 ceph-mon[32250]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:12:16.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:16 smithi019 ceph-mon[29213]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:12:16.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:16 smithi169 ceph-mon[32710]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:12:16.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:16 smithi150 ceph-mon[32250]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:12:18.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:18 smithi019 ceph-mon[29213]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:12:18.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:18 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:12:18.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:18 smithi169 ceph-mon[32710]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:12:18.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:18 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:12:18.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:18 smithi150 ceph-mon[32250]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:12:18.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:18 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:12:19.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:19 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:12:19.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:19 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:12:19.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:19 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:12:19.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:19 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:12:19.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:19 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:12:19.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:19 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:12:19.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:19 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:12:19.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:19 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:12:19.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:19 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:12:20.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:20 smithi019 ceph-mon[29213]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:12:20.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:20 smithi169 ceph-mon[32710]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:12:20.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:20 smithi150 ceph-mon[32250]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:12:22.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:22 smithi019 ceph-mon[29213]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:12:22.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:22 smithi169 ceph-mon[32710]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:12:22.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:22 smithi150 ceph-mon[32250]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:12:24.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:24 smithi019 ceph-mon[29213]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:24.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:24 smithi169 ceph-mon[32710]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:24.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:24 smithi150 ceph-mon[32250]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:26.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:26 smithi019 ceph-mon[29213]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:26.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:26 smithi169 ceph-mon[32710]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:26.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:26 smithi150 ceph-mon[32250]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:28.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:28 smithi019 ceph-mon[29213]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:28.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:28 smithi169 ceph-mon[32710]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:28.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:28 smithi150 ceph-mon[32250]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:30.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:30 smithi019 ceph-mon[29213]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:30.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:30 smithi169 ceph-mon[32710]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:30.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:30 smithi150 ceph-mon[32250]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:32.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:32 smithi019 ceph-mon[29213]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:32.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:32 smithi169 ceph-mon[32710]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:32.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:32 smithi150 ceph-mon[32250]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:34.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:34 smithi019 ceph-mon[29213]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:34.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:34 smithi169 ceph-mon[32710]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:34.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:34 smithi150 ceph-mon[32250]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:36.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:36 smithi019 ceph-mon[29213]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:36.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:36 smithi169 ceph-mon[32710]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:36.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:36 smithi150 ceph-mon[32250]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:38.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:38 smithi019 ceph-mon[29213]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:38.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:38 smithi169 ceph-mon[32710]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:38.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:38 smithi150 ceph-mon[32250]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:40.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:40 smithi019 ceph-mon[29213]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:40.657 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:40 smithi169 ceph-mon[32710]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:40.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:40 smithi150 ceph-mon[32250]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:42.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:42 smithi169 ceph-mon[32710]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:42.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:42 smithi150 ceph-mon[32250]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:42.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:42 smithi019 ceph-mon[29213]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:44.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:44 smithi169 ceph-mon[32710]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:44.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:44 smithi150 ceph-mon[32250]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:44.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:44 smithi019 ceph-mon[29213]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:46.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:46 smithi169 ceph-mon[32710]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:46.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:46 smithi150 ceph-mon[32250]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:46.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:46 smithi019 ceph-mon[29213]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:48.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:48 smithi019 ceph-mon[29213]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:48.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:48 smithi169 ceph-mon[32710]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:48.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:48 smithi150 ceph-mon[32250]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:50.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:50 smithi169 ceph-mon[32710]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:50.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:50 smithi150 ceph-mon[32250]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:50.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:50 smithi019 ceph-mon[29213]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:52.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:52 smithi169 ceph-mon[32710]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:52.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:52 smithi150 ceph-mon[32250]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:52.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:52 smithi019 ceph-mon[29213]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:54.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:54 smithi169 ceph-mon[32710]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:54.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:54 smithi150 ceph-mon[32250]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:54.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:54 smithi019 ceph-mon[29213]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:56.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:56 smithi169 ceph-mon[32710]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:56.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:56 smithi150 ceph-mon[32250]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:56.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:56 smithi019 ceph-mon[29213]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:12:58.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:12:58 smithi169 ceph-mon[32710]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:58.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:12:58 smithi150 ceph-mon[32250]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:12:58.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:12:58 smithi019 ceph-mon[29213]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:00.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:00 smithi169 ceph-mon[32710]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:00.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:00 smithi150 ceph-mon[32250]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:00.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:00 smithi019 ceph-mon[29213]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:02.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:02 smithi169 ceph-mon[32710]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:02.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:02 smithi150 ceph-mon[32250]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:02.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:02 smithi019 ceph-mon[29213]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:04.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:04 smithi169 ceph-mon[32710]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:04.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:04 smithi150 ceph-mon[32250]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:04.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:04 smithi019 ceph-mon[29213]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:06.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:06 smithi169 ceph-mon[32710]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:13:06.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:06 smithi150 ceph-mon[32250]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:13:06.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:06 smithi019 ceph-mon[29213]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:13:08.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:08 smithi169 ceph-mon[32710]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:13:08.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:08 smithi150 ceph-mon[32250]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:13:08.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:08 smithi019 ceph-mon[29213]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:13:10.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:10 smithi169 ceph-mon[32710]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:13:10.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:10 smithi150 ceph-mon[32250]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:13:10.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:10 smithi019 ceph-mon[29213]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:13:12.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:12 smithi169 ceph-mon[32710]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:13:12.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:12 smithi150 ceph-mon[32250]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:13:12.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:12 smithi019 ceph-mon[29213]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:13:14.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:14 smithi169 ceph-mon[32710]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:13:14.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:14 smithi150 ceph-mon[32250]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:13:14.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:14 smithi019 ceph-mon[29213]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T12:13:16.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:16 smithi169 ceph-mon[32710]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:13:16.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:16 smithi150 ceph-mon[32250]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:13:16.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:16 smithi019 ceph-mon[29213]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T12:13:18.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:18 smithi169 ceph-mon[32710]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:13:18.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:18 smithi150 ceph-mon[32250]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:13:18.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:18 smithi019 ceph-mon[29213]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:13:19.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:19 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:13:19.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:19 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:13:19.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:19 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:13:20.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:20 smithi169 ceph-mon[32710]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:13:20.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:20 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:13:20.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:20 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:13:20.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:20 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:13:20.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:20 smithi150 ceph-mon[32250]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:13:20.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:20 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:13:20.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:20 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:13:20.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:20 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:13:20.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:20 smithi019 ceph-mon[29213]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T12:13:20.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:20 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:13:20.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:20 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:13:20.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:20 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:13:22.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:22 smithi169 ceph-mon[32710]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:13:22.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:22 smithi150 ceph-mon[32250]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:13:22.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:22 smithi019 ceph-mon[29213]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T12:13:24.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:24 smithi169 ceph-mon[32710]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:24.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:24 smithi150 ceph-mon[32250]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:24.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:24 smithi019 ceph-mon[29213]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:26.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:26 smithi169 ceph-mon[32710]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:26.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:26 smithi150 ceph-mon[32250]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:26.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:26 smithi019 ceph-mon[29213]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:28.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:28 smithi169 ceph-mon[32710]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:28.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:28 smithi150 ceph-mon[32250]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:28.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:28 smithi019 ceph-mon[29213]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:30.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:30 smithi169 ceph-mon[32710]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:30.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:30 smithi150 ceph-mon[32250]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:30.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:30 smithi019 ceph-mon[29213]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:32.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:32 smithi169 ceph-mon[32710]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:32.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:32 smithi150 ceph-mon[32250]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:32.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:32 smithi019 ceph-mon[29213]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:34.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:34 smithi169 ceph-mon[32710]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:34.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:34 smithi150 ceph-mon[32250]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:34.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:34 smithi019 ceph-mon[29213]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:36.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:36 smithi169 ceph-mon[32710]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:36.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:36 smithi150 ceph-mon[32250]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:36.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:36 smithi019 ceph-mon[29213]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:38.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:38 smithi169 ceph-mon[32710]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:38.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:38 smithi150 ceph-mon[32250]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:38.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:38 smithi019 ceph-mon[29213]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:40.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:40 smithi150 ceph-mon[32250]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:40.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:40 smithi019 ceph-mon[29213]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:40.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:40 smithi169 ceph-mon[32710]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:42.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:42 smithi150 ceph-mon[32250]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:42.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:42 smithi019 ceph-mon[29213]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:42.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:42 smithi169 ceph-mon[32710]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:44.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:44 smithi150 ceph-mon[32250]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:44.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:44 smithi019 ceph-mon[29213]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:44.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:44 smithi169 ceph-mon[32710]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:46.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:46 smithi019 ceph-mon[29213]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:46.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:46 smithi169 ceph-mon[32710]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:46.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:46 smithi150 ceph-mon[32250]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:48.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:48 smithi019 ceph-mon[29213]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:48.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:48 smithi169 ceph-mon[32710]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:48.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:48 smithi150 ceph-mon[32250]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:50.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:50 smithi019 ceph-mon[29213]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:50.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:50 smithi169 ceph-mon[32710]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:50.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:50 smithi150 ceph-mon[32250]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:52.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:52 smithi019 ceph-mon[29213]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:52.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:52 smithi169 ceph-mon[32710]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:52.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:52 smithi150 ceph-mon[32250]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:54.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:54 smithi019 ceph-mon[29213]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:54.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:54 smithi169 ceph-mon[32710]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:54.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:54 smithi150 ceph-mon[32250]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:56.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:56 smithi019 ceph-mon[29213]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:56.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:56 smithi169 ceph-mon[32710]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:56.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:56 smithi150 ceph-mon[32250]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:13:58.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:13:58 smithi019 ceph-mon[29213]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:58.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:13:58 smithi169 ceph-mon[32710]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:13:58.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:13:58 smithi150 ceph-mon[32250]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:00.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:00 smithi019 ceph-mon[29213]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:00.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:00 smithi169 ceph-mon[32710]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:00.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:00 smithi150 ceph-mon[32250]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:02.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:02 smithi019 ceph-mon[29213]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-04-25T12:14:02.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:02 smithi169 ceph-mon[32710]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-04-25T12:14:02.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:02 smithi150 ceph-mon[32250]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 KiB/s wr, 28 op/s 2024-04-25T12:14:04.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:04 smithi019 ceph-mon[29213]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-04-25T12:14:04.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:04 smithi169 ceph-mon[32710]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-04-25T12:14:04.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:04 smithi150 ceph-mon[32250]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-04-25T12:14:06.710 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:06 smithi169 ceph-mon[32710]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T12:14:06.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:06 smithi019 ceph-mon[29213]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T12:14:06.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:06 smithi150 ceph-mon[32250]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T12:14:08.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:08 smithi019 ceph-mon[29213]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:14:08.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:08 smithi169 ceph-mon[32710]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:14:08.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:08 smithi150 ceph-mon[32250]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:14:10.718 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:10 smithi150 ceph-mon[32250]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:14:10.718 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:10 smithi150 ceph-mon[32250]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T12:14:10.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:10 smithi019 ceph-mon[29213]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:14:10.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:10 smithi019 ceph-mon[29213]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T12:14:10.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:10 smithi169 ceph-mon[32710]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:14:10.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:10 smithi169 ceph-mon[32710]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T12:14:12.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:12 smithi019 ceph-mon[29213]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T12:14:12.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:12 smithi169 ceph-mon[32710]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T12:14:12.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:12 smithi150 ceph-mon[32250]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T12:14:14.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:14 smithi019 ceph-mon[29213]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T12:14:14.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:14 smithi169 ceph-mon[32710]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T12:14:14.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:14 smithi150 ceph-mon[32250]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T12:14:16.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:16 smithi169 ceph-mon[32710]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T12:14:16.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:16 smithi169 ceph-mon[32710]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T12:14:16.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:16 smithi150 ceph-mon[32250]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T12:14:16.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:16 smithi150 ceph-mon[32250]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T12:14:16.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:16 smithi019 ceph-mon[29213]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T12:14:16.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:16 smithi019 ceph-mon[29213]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T12:14:18.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:18 smithi019 ceph-mon[29213]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T12:14:18.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:18 smithi169 ceph-mon[32710]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T12:14:18.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:18 smithi150 ceph-mon[32250]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T12:14:20.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:20 smithi169 ceph-mon[32710]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 210 B/s wr, 10 op/s 2024-04-25T12:14:20.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:20 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:14:20.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:20 smithi150 ceph-mon[32250]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 210 B/s wr, 10 op/s 2024-04-25T12:14:20.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:20 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:14:20.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:20 smithi019 ceph-mon[29213]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 210 B/s wr, 10 op/s 2024-04-25T12:14:20.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:20 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:14:21.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:21 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:14:21.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:21 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:14:21.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:21 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:14:21.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:21 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:14:21.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:21 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:14:21.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:21 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:14:21.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:21 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:14:21.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:21 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:14:21.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:21 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:14:22.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:22 smithi019 ceph-mon[29213]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:14:22.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:22 smithi169 ceph-mon[32710]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:14:22.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:22 smithi150 ceph-mon[32250]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:14:24.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:24 smithi019 ceph-mon[29213]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:14:24.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:24 smithi169 ceph-mon[32710]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:14:24.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:24 smithi150 ceph-mon[32250]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:14:26.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:26 smithi019 ceph-mon[29213]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:14:26.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:26 smithi169 ceph-mon[32710]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:14:26.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:26 smithi150 ceph-mon[32250]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:14:28.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:28 smithi019 ceph-mon[29213]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:28.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:28 smithi169 ceph-mon[32710]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:28.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:28 smithi150 ceph-mon[32250]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:30.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:30 smithi019 ceph-mon[29213]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:30.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:30 smithi169 ceph-mon[32710]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:30.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:30 smithi150 ceph-mon[32250]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:32.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:32 smithi169 ceph-mon[32710]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:32.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:32 smithi150 ceph-mon[32250]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:33.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:32 smithi019 ceph-mon[29213]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:34.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:34 smithi169 ceph-mon[32710]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:34.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:34 smithi150 ceph-mon[32250]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:35.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:34 smithi019 ceph-mon[29213]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:36.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:36 smithi169 ceph-mon[32710]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:36.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:36 smithi150 ceph-mon[32250]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:37.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:36 smithi019 ceph-mon[29213]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:38.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:38 smithi169 ceph-mon[32710]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:38.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:38 smithi150 ceph-mon[32250]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:39.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:38 smithi019 ceph-mon[29213]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:40.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:40 smithi169 ceph-mon[32710]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:40.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:40 smithi150 ceph-mon[32250]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:41.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:40 smithi019 ceph-mon[29213]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:42.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:42 smithi169 ceph-mon[32710]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:42.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:42 smithi150 ceph-mon[32250]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:43.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:42 smithi019 ceph-mon[29213]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:44.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:44 smithi169 ceph-mon[32710]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:14:44.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:44 smithi150 ceph-mon[32250]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:14:45.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:44 smithi019 ceph-mon[29213]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:14:46.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:46 smithi169 ceph-mon[32710]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:46.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:46 smithi150 ceph-mon[32250]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:47.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:46 smithi019 ceph-mon[29213]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:48.852 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:48 smithi019 ceph-mon[29213]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:14:48.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:48 smithi169 ceph-mon[32710]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:14:48.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:48 smithi150 ceph-mon[32250]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:14:50.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:50 smithi169 ceph-mon[32710]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:14:50.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:50 smithi150 ceph-mon[32250]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:14:51.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:50 smithi019 ceph-mon[29213]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:14:52.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:52 smithi169 ceph-mon[32710]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:52.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:52 smithi150 ceph-mon[32250]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:53.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:52 smithi019 ceph-mon[29213]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:54.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:54 smithi169 ceph-mon[32710]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:54.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:54 smithi150 ceph-mon[32250]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:55.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:54 smithi019 ceph-mon[29213]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:56.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:56 smithi169 ceph-mon[32710]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:56.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:56 smithi150 ceph-mon[32250]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:57.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:56 smithi019 ceph-mon[29213]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:14:58.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:14:58 smithi169 ceph-mon[32710]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:58.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:14:58 smithi150 ceph-mon[32250]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:14:59.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:14:58 smithi019 ceph-mon[29213]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:00.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:00 smithi169 ceph-mon[32710]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:00.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:00 smithi150 ceph-mon[32250]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:01.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:00 smithi019 ceph-mon[29213]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:02.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:02 smithi169 ceph-mon[32710]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T12:15:02.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:02 smithi150 ceph-mon[32250]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T12:15:03.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:02 smithi019 ceph-mon[29213]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T12:15:04.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:04 smithi169 ceph-mon[32710]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T12:15:04.915 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:04 smithi150 ceph-mon[32250]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T12:15:05.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:04 smithi019 ceph-mon[29213]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T12:15:06.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:06 smithi169 ceph-mon[32710]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T12:15:06.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:06 smithi150 ceph-mon[32250]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T12:15:07.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:06 smithi019 ceph-mon[29213]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T12:15:08.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:08 smithi169 ceph-mon[32710]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:15:08.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:08 smithi150 ceph-mon[32250]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:15:09.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:08 smithi019 ceph-mon[29213]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:15:10.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:10 smithi169 ceph-mon[32710]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:15:10.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:10 smithi169 ceph-mon[32710]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T12:15:10.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:10 smithi150 ceph-mon[32250]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:15:10.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:10 smithi150 ceph-mon[32250]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T12:15:11.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:10 smithi019 ceph-mon[29213]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:15:11.051 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:10 smithi019 ceph-mon[29213]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T12:15:12.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:12 smithi169 ceph-mon[32710]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-25T12:15:12.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:12 smithi150 ceph-mon[32250]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-25T12:15:13.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:12 smithi019 ceph-mon[29213]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-25T12:15:14.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:14 smithi169 ceph-mon[32710]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-25T12:15:14.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:14 smithi150 ceph-mon[32250]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-25T12:15:15.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:14 smithi019 ceph-mon[29213]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-25T12:15:16.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:16 smithi169 ceph-mon[32710]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T12:15:16.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:16 smithi169 ceph-mon[32710]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T12:15:16.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:16 smithi150 ceph-mon[32250]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T12:15:16.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:16 smithi150 ceph-mon[32250]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T12:15:16.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:16 smithi019 ceph-mon[29213]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T12:15:16.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:16 smithi019 ceph-mon[29213]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T12:15:18.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:18 smithi019 ceph-mon[29213]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T12:15:18.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:18 smithi169 ceph-mon[32710]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T12:15:18.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:18 smithi150 ceph-mon[32250]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T12:15:20.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:20 smithi169 ceph-mon[32710]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 213 B/s wr, 10 op/s 2024-04-25T12:15:20.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:20 smithi150 ceph-mon[32250]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 213 B/s wr, 10 op/s 2024-04-25T12:15:20.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:20 smithi019 ceph-mon[29213]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 213 B/s wr, 10 op/s 2024-04-25T12:15:21.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:21 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:15:21.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:21 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:15:21.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:21 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:15:21.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:21 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:15:21.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:21 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:15:21.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:21 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:15:21.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:21 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:15:21.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:21 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:15:21.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:21 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:15:21.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:21 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:15:21.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:21 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:15:21.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:21 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:15:22.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:22 smithi169 ceph-mon[32710]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:15:22.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:22 smithi150 ceph-mon[32250]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:15:22.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:22 smithi019 ceph-mon[29213]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:15:24.662 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:24 smithi169 ceph-mon[32710]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:15:24.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:24 smithi150 ceph-mon[32250]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:15:24.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:24 smithi019 ceph-mon[29213]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:15:26.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:26 smithi169 ceph-mon[32710]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:15:26.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:26 smithi150 ceph-mon[32250]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:15:26.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:26 smithi019 ceph-mon[29213]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:15:28.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:28 smithi169 ceph-mon[32710]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:28.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:28 smithi150 ceph-mon[32250]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:28.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:28 smithi019 ceph-mon[29213]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:30.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:30 smithi169 ceph-mon[32710]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:30.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:30 smithi150 ceph-mon[32250]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:30.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:30 smithi019 ceph-mon[29213]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:32.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:32 smithi169 ceph-mon[32710]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:32.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:32 smithi150 ceph-mon[32250]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:32.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:32 smithi019 ceph-mon[29213]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:34.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:34 smithi169 ceph-mon[32710]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:34.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:34 smithi150 ceph-mon[32250]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:34.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:34 smithi019 ceph-mon[29213]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:36.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:36 smithi169 ceph-mon[32710]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:36.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:36 smithi150 ceph-mon[32250]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:36.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:36 smithi019 ceph-mon[29213]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:38.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:38 smithi169 ceph-mon[32710]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:38.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:38 smithi150 ceph-mon[32250]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:38.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:38 smithi019 ceph-mon[29213]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:40.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:40 smithi169 ceph-mon[32710]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:40.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:40 smithi150 ceph-mon[32250]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:40.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:40 smithi019 ceph-mon[29213]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:42.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:42 smithi169 ceph-mon[32710]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:42.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:42 smithi150 ceph-mon[32250]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:42.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:42 smithi019 ceph-mon[29213]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:44.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:44 smithi169 ceph-mon[32710]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:44.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:44 smithi150 ceph-mon[32250]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:44.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:44 smithi019 ceph-mon[29213]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:46.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:46 smithi169 ceph-mon[32710]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:46.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:46 smithi150 ceph-mon[32250]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:46.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:46 smithi019 ceph-mon[29213]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:48.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:48 smithi169 ceph-mon[32710]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:48.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:48 smithi150 ceph-mon[32250]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:48.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:48 smithi019 ceph-mon[29213]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:50.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:50 smithi169 ceph-mon[32710]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:50.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:50 smithi150 ceph-mon[32250]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:50.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:50 smithi019 ceph-mon[29213]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:52.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:52 smithi169 ceph-mon[32710]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:52.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:52 smithi150 ceph-mon[32250]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:52.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:52 smithi019 ceph-mon[29213]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:54.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:54 smithi169 ceph-mon[32710]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:54.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:54 smithi150 ceph-mon[32250]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:54.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:54 smithi019 ceph-mon[29213]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:56.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:56 smithi169 ceph-mon[32710]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:56.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:56 smithi150 ceph-mon[32250]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:56.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:56 smithi019 ceph-mon[29213]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:15:58.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:15:58 smithi169 ceph-mon[32710]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:58.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:15:58 smithi150 ceph-mon[32250]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:15:58.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:15:58 smithi019 ceph-mon[29213]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:00.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:00 smithi169 ceph-mon[32710]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:00.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:00 smithi150 ceph-mon[32250]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:00.803 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:00 smithi019 ceph-mon[29213]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:02.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:02 smithi169 ceph-mon[32710]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:02.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:02 smithi150 ceph-mon[32250]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:02.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:02 smithi019 ceph-mon[29213]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:04.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:04 smithi169 ceph-mon[32710]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:04.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:04 smithi150 ceph-mon[32250]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:04.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:04 smithi019 ceph-mon[29213]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:06.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:06 smithi169 ceph-mon[32710]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T12:16:06.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:06 smithi150 ceph-mon[32250]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T12:16:06.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:06 smithi019 ceph-mon[29213]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T12:16:08.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:08 smithi150 ceph-mon[32250]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:16:08.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:08 smithi019 ceph-mon[29213]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:16:08.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:08 smithi169 ceph-mon[32710]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:16:10.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:10 smithi150 ceph-mon[32250]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:16:10.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:10 smithi150 ceph-mon[32250]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T12:16:10.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:10 smithi019 ceph-mon[29213]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:16:10.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:10 smithi019 ceph-mon[29213]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T12:16:10.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:10 smithi169 ceph-mon[32710]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:16:10.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:10 smithi169 ceph-mon[32710]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T12:16:12.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:12 smithi019 ceph-mon[29213]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T12:16:12.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:12 smithi169 ceph-mon[32710]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T12:16:12.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:12 smithi150 ceph-mon[32250]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T12:16:14.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:14 smithi019 ceph-mon[29213]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T12:16:14.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:14 smithi169 ceph-mon[32710]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T12:16:14.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:14 smithi150 ceph-mon[32250]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T12:16:16.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:16 smithi169 ceph-mon[32710]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T12:16:16.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:16 smithi169 ceph-mon[32710]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T12:16:16.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:16 smithi150 ceph-mon[32250]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T12:16:16.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:16 smithi150 ceph-mon[32250]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T12:16:16.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:16 smithi019 ceph-mon[29213]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T12:16:16.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:16 smithi019 ceph-mon[29213]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T12:16:18.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:18 smithi019 ceph-mon[29213]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T12:16:18.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:18 smithi169 ceph-mon[32710]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T12:16:18.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:18 smithi150 ceph-mon[32250]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T12:16:20.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:20 smithi169 ceph-mon[32710]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 208 B/s wr, 10 op/s 2024-04-25T12:16:20.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:20 smithi150 ceph-mon[32250]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 208 B/s wr, 10 op/s 2024-04-25T12:16:20.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:20 smithi019 ceph-mon[29213]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 208 B/s wr, 10 op/s 2024-04-25T12:16:22.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:22 smithi169 ceph-mon[32710]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:16:22.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:22 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:16:22.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:22 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:16:22.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:22 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:16:22.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:22 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:16:22.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:22 smithi150 ceph-mon[32250]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:16:22.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:22 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:16:22.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:22 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:16:22.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:22 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:16:22.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:22 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:16:22.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:22 smithi019 ceph-mon[29213]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:16:22.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:22 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:16:22.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:22 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:16:22.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:22 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:16:22.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:22 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:16:24.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:24 smithi169 ceph-mon[32710]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:16:24.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:24 smithi150 ceph-mon[32250]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:16:24.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:24 smithi019 ceph-mon[29213]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T12:16:26.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:26 smithi169 ceph-mon[32710]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:16:26.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:26 smithi150 ceph-mon[32250]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:16:26.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:26 smithi019 ceph-mon[29213]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:16:28.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:28 smithi169 ceph-mon[32710]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:28.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:28 smithi150 ceph-mon[32250]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:28.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:28 smithi019 ceph-mon[29213]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:30.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:30 smithi169 ceph-mon[32710]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:30.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:30 smithi150 ceph-mon[32250]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:30.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:30 smithi019 ceph-mon[29213]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:32.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:32 smithi169 ceph-mon[32710]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:32.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:32 smithi150 ceph-mon[32250]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:32.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:32 smithi019 ceph-mon[29213]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:34.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:34 smithi169 ceph-mon[32710]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:16:34.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:34 smithi150 ceph-mon[32250]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:16:34.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:34 smithi019 ceph-mon[29213]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:16:36.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:36 smithi169 ceph-mon[32710]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:36.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:36 smithi150 ceph-mon[32250]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:36.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:36 smithi019 ceph-mon[29213]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:38.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:38 smithi169 ceph-mon[32710]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:16:38.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:38 smithi150 ceph-mon[32250]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:16:38.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:38 smithi019 ceph-mon[29213]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:16:40.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:40 smithi019 ceph-mon[29213]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:16:40.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:40 smithi169 ceph-mon[32710]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:16:40.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:40 smithi150 ceph-mon[32250]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T12:16:42.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:42 smithi169 ceph-mon[32710]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:42.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:42 smithi150 ceph-mon[32250]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:42.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:42 smithi019 ceph-mon[29213]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:44.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:44 smithi169 ceph-mon[32710]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:44.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:44 smithi150 ceph-mon[32250]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:44.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:44 smithi019 ceph-mon[29213]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:46.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:46 smithi169 ceph-mon[32710]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:46.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:46 smithi150 ceph-mon[32250]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:46.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:46 smithi019 ceph-mon[29213]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:48.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:48 smithi169 ceph-mon[32710]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:48.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:48 smithi150 ceph-mon[32250]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:48.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:48 smithi019 ceph-mon[29213]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:50.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:50 smithi169 ceph-mon[32710]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:50.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:50 smithi150 ceph-mon[32250]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:50.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:50 smithi019 ceph-mon[29213]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:52.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:52 smithi169 ceph-mon[32710]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:52.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:52 smithi150 ceph-mon[32250]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:52.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:52 smithi019 ceph-mon[29213]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:54.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:54 smithi169 ceph-mon[32710]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:54.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:54 smithi150 ceph-mon[32250]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:54.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:54 smithi019 ceph-mon[29213]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:56.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:56 smithi169 ceph-mon[32710]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:56.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:56 smithi150 ceph-mon[32250]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:56.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:56 smithi019 ceph-mon[29213]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:16:58.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:16:58 smithi169 ceph-mon[32710]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:58.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:16:58 smithi150 ceph-mon[32250]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:16:58.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:16:58 smithi019 ceph-mon[29213]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:00.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:00 smithi169 ceph-mon[32710]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:00.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:00 smithi150 ceph-mon[32250]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:00.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:00 smithi019 ceph-mon[29213]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:02.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:02 smithi150 ceph-mon[32250]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:02.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:02 smithi019 ceph-mon[29213]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:02.909 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:02 smithi169 ceph-mon[32710]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:04.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:04 smithi150 ceph-mon[32250]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:04.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:04 smithi019 ceph-mon[29213]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:04.909 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:04 smithi169 ceph-mon[32710]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:06.711 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:06 smithi169 ceph-mon[32710]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T12:17:06.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:06 smithi019 ceph-mon[29213]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T12:17:06.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:06 smithi150 ceph-mon[32250]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T12:17:08.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:08 smithi019 ceph-mon[29213]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:17:08.909 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:08 smithi169 ceph-mon[32710]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:17:08.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:08 smithi150 ceph-mon[32250]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:17:10.718 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:10 smithi150 ceph-mon[32250]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:17:10.719 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:10 smithi150 ceph-mon[32250]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T12:17:10.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:10 smithi019 ceph-mon[29213]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:17:10.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:10 smithi019 ceph-mon[29213]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T12:17:10.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:10 smithi169 ceph-mon[32710]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T12:17:10.909 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:10 smithi169 ceph-mon[32710]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T12:17:11.862 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T12:17:11.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.979+0000 7ff0fa15d640 1 -- 172.21.15.19:0/881862778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f406a780 msgr2=0x7ff0f406abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:11.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.979+0000 7ff0fa15d640 1 --2- 172.21.15.19:0/881862778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f406a780 0x7ff0f406abe0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009900 tx=0x7ff0e802a480 comp rx=0 tx=0).stop 2024-04-25T12:17:11.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.979+0000 7ff0fa15d640 1 -- 172.21.15.19:0/881862778 shutdown_connections 2024-04-25T12:17:11.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.979+0000 7ff0fa15d640 1 --2- 172.21.15.19:0/881862778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0f406b120 0x7ff0f410edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.979+0000 7ff0fa15d640 1 --2- 172.21.15.19:0/881862778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f406a780 0x7ff0f406abe0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.979+0000 7ff0fa15d640 1 --2- 172.21.15.19:0/881862778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0f41080e0 0x7ff0f41084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.979+0000 7ff0fa15d640 1 -- 172.21.15.19:0/881862778 >> 172.21.15.19:0/881862778 conn(0x7ff0f4076c50 msgr2=0x7ff0f4077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.979+0000 7ff0fa15d640 1 -- 172.21.15.19:0/881862778 shutdown_connections 2024-04-25T12:17:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.979+0000 7ff0fa15d640 1 -- 172.21.15.19:0/881862778 wait complete. 2024-04-25T12:17:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.980+0000 7ff0fa15d640 1 Processor -- start 2024-04-25T12:17:11.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.980+0000 7ff0fa15d640 1 -- start start 2024-04-25T12:17:11.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.980+0000 7ff0fa15d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0f406a780 0x7ff0f4101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:11.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0fa15d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f406b120 0x7ff0f41017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:11.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0fa15d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0f41080e0 0x7ff0f4101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:11.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0fa15d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f41122d0 con 0x7ff0f406b120 2024-04-25T12:17:11.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0fa15d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4112150 con 0x7ff0f41080e0 2024-04-25T12:17:11.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0fa15d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4112450 con 0x7ff0f406a780 2024-04-25T12:17:11.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0f2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f406b120 0x7ff0f41017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:11.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0f2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f406b120 0x7ff0f41017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37368/0 (socket says 172.21.15.19:37368) 2024-04-25T12:17:11.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0f2ffd640 1 -- 172.21.15.19:0/4131806972 learned_addr learned my addr 172.21.15.19:0/4131806972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:11.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0f3fff640 1 --2- 172.21.15.19:0/4131806972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0f41080e0 0x7ff0f4101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:11.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0f37fe640 1 --2- 172.21.15.19:0/4131806972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0f406a780 0x7ff0f4101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:11.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0f2ffd640 1 -- 172.21.15.19:0/4131806972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0f406a780 msgr2=0x7ff0f4101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:11.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0f2ffd640 1 --2- 172.21.15.19:0/4131806972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0f406a780 0x7ff0f4101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:11.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0f2ffd640 1 -- 172.21.15.19:0/4131806972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0f41080e0 msgr2=0x7ff0f4101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:11.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0f2ffd640 1 --2- 172.21.15.19:0/4131806972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0f41080e0 0x7ff0f4101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:11.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0f2ffd640 1 -- 172.21.15.19:0/4131806972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f41a6bb0 con 0x7ff0f406b120 2024-04-25T12:17:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0f2ffd640 1 --2- 172.21.15.19:0/4131806972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f406b120 0x7ff0f41017b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8005ab0 tx=0x7ff0e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0f0ff9640 1 -- 172.21.15.19:0/4131806972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e803f030 con 0x7ff0f406b120 2024-04-25T12:17:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0f0ff9640 1 -- 172.21.15.19:0/4131806972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7ff0e8003cd0 con 0x7ff0f406b120 2024-04-25T12:17:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0fa15d640 1 -- 172.21.15.19:0/4131806972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f41a6e40 con 0x7ff0f406b120 2024-04-25T12:17:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.981+0000 7ff0f0ff9640 1 -- 172.21.15.19:0/4131806972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8035070 con 0x7ff0f406b120 2024-04-25T12:17:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.982+0000 7ff0fa15d640 1 -- 172.21.15.19:0/4131806972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f41a72a0 con 0x7ff0f406b120 2024-04-25T12:17:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.983+0000 7ff0fa15d640 1 -- 172.21.15.19:0/4131806972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f4103660 con 0x7ff0f406b120 2024-04-25T12:17:11.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.984+0000 7ff0f0ff9640 1 -- 172.21.15.19:0/4131806972 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff0e8042030 con 0x7ff0f406b120 2024-04-25T12:17:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.985+0000 7ff0f0ff9640 1 --2- 172.21.15.19:0/4131806972 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7ff0dc0a3dc0 0x7ff0dc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.985+0000 7ff0f37fe640 1 --2- 172.21.15.19:0/4131806972 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7ff0dc0a3dc0 0x7ff0dc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:11.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.985+0000 7ff0f0ff9640 1 -- 172.21.15.19:0/4131806972 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7ff0e80f10f0 con 0x7ff0f406b120 2024-04-25T12:17:11.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.985+0000 7ff0f37fe640 1 --2- 172.21.15.19:0/4131806972 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7ff0dc0a3dc0 0x7ff0dc0a6280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400cac0 tx=0x7ff0e400c4a0 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:11.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:11.990+0000 7ff0f0ff9640 1 -- 172.21.15.19:0/4131806972 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff0e80bd730 con 0x7ff0f406b120 2024-04-25T12:17:12.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.198+0000 7ff0fa15d640 1 -- 172.21.15.19:0/4131806972 --> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ff0f4100330 con 0x7ff0dc0a3dc0 2024-04-25T12:17:12.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.206+0000 7ff0f0ff9640 1 -- 172.21.15.19:0/4131806972 <== mgr.15381 v2:172.21.15.19:6800/3657854393 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7ff0f4100330 con 0x7ff0dc0a3dc0 2024-04-25T12:17:12.208 INFO:teuthology.orchestra.run.smithi019.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T12:07:10", "first": "2024-04-25T12:08:00", "last": "2024-04-25T12:17:00", "last_pruned": "2024-04-25T12:17:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T12:17:12.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.208+0000 7ff0fa15d640 1 -- 172.21.15.19:0/4131806972 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7ff0dc0a3dc0 msgr2=0x7ff0dc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.208+0000 7ff0fa15d640 1 --2- 172.21.15.19:0/4131806972 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7ff0dc0a3dc0 0x7ff0dc0a6280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400cac0 tx=0x7ff0e400c4a0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.208+0000 7ff0fa15d640 1 -- 172.21.15.19:0/4131806972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f406b120 msgr2=0x7ff0f41017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.208+0000 7ff0fa15d640 1 --2- 172.21.15.19:0/4131806972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f406b120 0x7ff0f41017b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8005ab0 tx=0x7ff0e80042f0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.208+0000 7ff0fa15d640 1 -- 172.21.15.19:0/4131806972 shutdown_connections 2024-04-25T12:17:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.208+0000 7ff0fa15d640 1 --2- 172.21.15.19:0/4131806972 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7ff0dc0a3dc0 0x7ff0dc0a6280 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.208+0000 7ff0fa15d640 1 --2- 172.21.15.19:0/4131806972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0f41080e0 0x7ff0f4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.208+0000 7ff0fa15d640 1 --2- 172.21.15.19:0/4131806972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f406b120 0x7ff0f41017b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.208+0000 7ff0fa15d640 1 --2- 172.21.15.19:0/4131806972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0f406a780 0x7ff0f4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.208+0000 7ff0fa15d640 1 -- 172.21.15.19:0/4131806972 >> 172.21.15.19:0/4131806972 conn(0x7ff0f4076c50 msgr2=0x7ff0f40ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:12.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.209+0000 7ff0fa15d640 1 -- 172.21.15.19:0/4131806972 shutdown_connections 2024-04-25T12:17:12.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.209+0000 7ff0fa15d640 1 -- 172.21.15.19:0/4131806972 wait complete. 2024-04-25T12:17:12.221 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T12:17:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.323+0000 7f1579064640 1 -- 172.21.15.19:0/3702793834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1574103200 msgr2=0x7f15741035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.323+0000 7f1579064640 1 --2- 172.21.15.19:0/3702793834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1574103200 0x7f15741035e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1564009900 tx=0x7f156402a480 comp rx=0 tx=0).stop 2024-04-25T12:17:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.323+0000 7f1579064640 1 -- 172.21.15.19:0/3702793834 shutdown_connections 2024-04-25T12:17:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.323+0000 7f1579064640 1 --2- 172.21.15.19:0/3702793834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f157410c5d0 0x7f157410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.323+0000 7f1579064640 1 --2- 172.21.15.19:0/3702793834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1574103b20 0x7f157410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.323+0000 7f1579064640 1 --2- 172.21.15.19:0/3702793834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1574103200 0x7f15741035e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.323+0000 7f1579064640 1 -- 172.21.15.19:0/3702793834 >> 172.21.15.19:0/3702793834 conn(0x7f15740771d0 msgr2=0x7f15740775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.323+0000 7f1579064640 1 -- 172.21.15.19:0/3702793834 shutdown_connections 2024-04-25T12:17:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.323+0000 7f1579064640 1 -- 172.21.15.19:0/3702793834 wait complete. 2024-04-25T12:17:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.324+0000 7f1579064640 1 Processor -- start 2024-04-25T12:17:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.324+0000 7f1579064640 1 -- start start 2024-04-25T12:17:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.324+0000 7f1579064640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1574103200 0x7f157419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.324+0000 7f1579064640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1574103b20 0x7f157419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.324+0000 7f1579064640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f157410c5d0 0x7f15741985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.324+0000 7f1579064640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1574112350 con 0x7f157410c5d0 2024-04-25T12:17:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.324+0000 7f1579064640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15741121d0 con 0x7f1574103200 2024-04-25T12:17:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.324+0000 7f1579064640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f15741124d0 con 0x7f1574103b20 2024-04-25T12:17:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f157359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f157410c5d0 0x7f15741985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f157359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f157410c5d0 0x7f15741985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37384/0 (socket says 172.21.15.19:37384) 2024-04-25T12:17:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f157359e640 1 -- 172.21.15.19:0/822223962 learned_addr learned my addr 172.21.15.19:0/822223962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f156bfff640 1 --2- 172.21.15.19:0/822223962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1574103b20 0x7f157419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f157359e640 1 -- 172.21.15.19:0/822223962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1574103b20 msgr2=0x7f157419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f157359e640 1 --2- 172.21.15.19:0/822223962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1574103b20 0x7f157419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f1572d9d640 1 --2- 172.21.15.19:0/822223962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1574103200 0x7f157419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f157359e640 1 -- 172.21.15.19:0/822223962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1574103200 msgr2=0x7f157419e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:12.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f157359e640 1 --2- 172.21.15.19:0/822223962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1574103200 0x7f157419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f157359e640 1 -- 172.21.15.19:0/822223962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1574198d50 con 0x7f157410c5d0 2024-04-25T12:17:12.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f157359e640 1 --2- 172.21.15.19:0/822223962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f157410c5d0 0x7f15741985d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f156c00ccb0 tx=0x7f156c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:12.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f1570d99640 1 -- 172.21.15.19:0/822223962 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c00f450 con 0x7f157410c5d0 2024-04-25T12:17:12.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f1570d99640 1 -- 172.21.15.19:0/822223962 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f156c00fac0 con 0x7f157410c5d0 2024-04-25T12:17:12.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f1579064640 1 -- 172.21.15.19:0/822223962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1574199040 con 0x7f157410c5d0 2024-04-25T12:17:12.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f1570d99640 1 -- 172.21.15.19:0/822223962 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c018bc0 con 0x7f157410c5d0 2024-04-25T12:17:12.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.325+0000 7f1579064640 1 -- 172.21.15.19:0/822223962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15741a52e0 con 0x7f157410c5d0 2024-04-25T12:17:12.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.326+0000 7f1579064640 1 -- 172.21.15.19:0/822223962 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f157404fce0 con 0x7f157410c5d0 2024-04-25T12:17:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.327+0000 7f1570d99640 1 -- 172.21.15.19:0/822223962 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f156c00fc70 con 0x7f157410c5d0 2024-04-25T12:17:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.328+0000 7f1570d99640 1 --2- 172.21.15.19:0/822223962 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f15600a3dc0 0x7f15600a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:12.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.328+0000 7f1572d9d640 1 --2- 172.21.15.19:0/822223962 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f15600a3dc0 0x7f15600a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:12.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.328+0000 7f1570d99640 1 -- 172.21.15.19:0/822223962 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7f156c0d0f40 con 0x7f157410c5d0 2024-04-25T12:17:12.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.328+0000 7f1572d9d640 1 --2- 172.21.15.19:0/822223962 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f15600a3dc0 0x7f15600a6280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f15640096c0 tx=0x7f1564000bc0 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:12.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.333+0000 7f1570d99640 1 -- 172.21.15.19:0/822223962 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f156c014030 con 0x7f157410c5d0 2024-04-25T12:17:12.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.544+0000 7f1579064640 1 -- 172.21.15.19:0/822223962 --> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f15740ff530 con 0x7f15600a3dc0 2024-04-25T12:17:12.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:12 smithi019 ceph-mon[29213]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T12:17:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.551+0000 7f1570d99640 1 -- 172.21.15.19:0/822223962 <== mgr.15381 v2:172.21.15.19:6800/3657854393 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f15740ff530 con 0x7f15600a3dc0 2024-04-25T12:17:12.553 INFO:teuthology.orchestra.run.smithi019.stdout:/ 1m 6m3h 2024-04-25T12:17:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.553+0000 7f1579064640 1 -- 172.21.15.19:0/822223962 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f15600a3dc0 msgr2=0x7f15600a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.553+0000 7f1579064640 1 --2- 172.21.15.19:0/822223962 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f15600a3dc0 0x7f15600a6280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f15640096c0 tx=0x7f1564000bc0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.553+0000 7f1579064640 1 -- 172.21.15.19:0/822223962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f157410c5d0 msgr2=0x7f15741985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.553+0000 7f1579064640 1 --2- 172.21.15.19:0/822223962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f157410c5d0 0x7f15741985d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f156c00ccb0 tx=0x7f156c00a620 comp rx=0 tx=0).stop 2024-04-25T12:17:12.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.553+0000 7f1579064640 1 -- 172.21.15.19:0/822223962 shutdown_connections 2024-04-25T12:17:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.553+0000 7f1579064640 1 --2- 172.21.15.19:0/822223962 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f15600a3dc0 0x7f15600a6280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.553+0000 7f1579064640 1 --2- 172.21.15.19:0/822223962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f157410c5d0 0x7f15741985d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.553+0000 7f1579064640 1 --2- 172.21.15.19:0/822223962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1574103b20 0x7f157419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.553+0000 7f1579064640 1 --2- 172.21.15.19:0/822223962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1574103200 0x7f157419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.553+0000 7f1579064640 1 -- 172.21.15.19:0/822223962 >> 172.21.15.19:0/822223962 conn(0x7f15740771d0 msgr2=0x7f15740fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.553+0000 7f1579064640 1 -- 172.21.15.19:0/822223962 shutdown_connections 2024-04-25T12:17:12.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.553+0000 7f1579064640 1 -- 172.21.15.19:0/822223962 wait complete. 2024-04-25T12:17:12.566 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T12:17:12.580 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:17:12.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.680+0000 7fb1b84fa640 1 -- 172.21.15.19:0/1507902688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1b0103b50 msgr2=0x7fb1b010c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.680+0000 7fb1b84fa640 1 --2- 172.21.15.19:0/1507902688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1b0103b50 0x7fb1b010c030 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009900 tx=0x7fb1a002a480 comp rx=0 tx=0).stop 2024-04-25T12:17:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.680+0000 7fb1b84fa640 1 -- 172.21.15.19:0/1507902688 shutdown_connections 2024-04-25T12:17:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.680+0000 7fb1b84fa640 1 --2- 172.21.15.19:0/1507902688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1b010c570 0x7fb1b010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.680+0000 7fb1b84fa640 1 --2- 172.21.15.19:0/1507902688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1b0103b50 0x7fb1b010c030 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.680+0000 7fb1b84fa640 1 --2- 172.21.15.19:0/1507902688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1b0103230 0x7fb1b0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.680+0000 7fb1b84fa640 1 -- 172.21.15.19:0/1507902688 >> 172.21.15.19:0/1507902688 conn(0x7fb1b0077fb0 msgr2=0x7fb1b00783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.680+0000 7fb1b84fa640 1 -- 172.21.15.19:0/1507902688 shutdown_connections 2024-04-25T12:17:12.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.680+0000 7fb1b84fa640 1 -- 172.21.15.19:0/1507902688 wait complete. 2024-04-25T12:17:12.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.680+0000 7fb1b84fa640 1 Processor -- start 2024-04-25T12:17:12.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b84fa640 1 -- start start 2024-04-25T12:17:12.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b84fa640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1b0103230 0x7fb1b0100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:12.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b84fa640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1b0103b50 0x7fb1b01013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:12.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b84fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1b010c570 0x7fb1b0101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:12.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b84fa640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b01123f0 con 0x7fb1b010c570 2024-04-25T12:17:12.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b84fa640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b0112270 con 0x7fb1b0103b50 2024-04-25T12:17:12.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b84fa640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b0112570 con 0x7fb1b0103230 2024-04-25T12:17:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b6a97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1b010c570 0x7fb1b0101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b6a97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1b010c570 0x7fb1b0101900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37402/0 (socket says 172.21.15.19:37402) 2024-04-25T12:17:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b6a97640 1 -- 172.21.15.19:0/296740801 learned_addr learned my addr 172.21.15.19:0/296740801 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b6296640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1b0103230 0x7fb1b0100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b5a95640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1b0103b50 0x7fb1b01013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b6a97640 1 -- 172.21.15.19:0/296740801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1b0103230 msgr2=0x7fb1b0100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b6a97640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1b0103230 0x7fb1b0100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b6a97640 1 -- 172.21.15.19:0/296740801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1b0103b50 msgr2=0x7fb1b01013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b6a97640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1b0103b50 0x7fb1b01013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b6a97640 1 -- 172.21.15.19:0/296740801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b01a6ed0 con 0x7fb1b010c570 2024-04-25T12:17:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b6296640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1b0103230 0x7fb1b0100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:17:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b5a95640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1b0103b50 0x7fb1b01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:17:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1b6a97640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1b010c570 0x7fb1b0101900 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800b9e0 tx=0x7fb1a800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.681+0000 7fb1a77fe640 1 -- 172.21.15.19:0/296740801 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8019070 con 0x7fb1b010c570 2024-04-25T12:17:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.682+0000 7fb1a77fe640 1 -- 172.21.15.19:0/296740801 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb1a8014440 con 0x7fb1b010c570 2024-04-25T12:17:12.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.682+0000 7fb1b84fa640 1 -- 172.21.15.19:0/296740801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b01a71c0 con 0x7fb1b010c570 2024-04-25T12:17:12.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.682+0000 7fb1a77fe640 1 -- 172.21.15.19:0/296740801 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80075d0 con 0x7fb1b010c570 2024-04-25T12:17:12.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.682+0000 7fb1b84fa640 1 -- 172.21.15.19:0/296740801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b01a7710 con 0x7fb1b010c570 2024-04-25T12:17:12.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.683+0000 7fb1b84fa640 1 -- 172.21.15.19:0/296740801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b00691c0 con 0x7fb1b010c570 2024-04-25T12:17:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.684+0000 7fb1a77fe640 1 -- 172.21.15.19:0/296740801 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb1a801c070 con 0x7fb1b010c570 2024-04-25T12:17:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.685+0000 7fb1a77fe640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb19c0a3d20 0x7fb19c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.685+0000 7fb1a77fe640 1 -- 172.21.15.19:0/296740801 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7fb1a80cfd60 con 0x7fb1b010c570 2024-04-25T12:17:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.685+0000 7fb1b6296640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb19c0a3d20 0x7fb19c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:12.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.686+0000 7fb1b6296640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb19c0a3d20 0x7fb19c0a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009930 tx=0x7fb1ac008040 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:12.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.691+0000 7fb1a77fe640 1 -- 172.21.15.19:0/296740801 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb1a809c410 con 0x7fb1b010c570 2024-04-25T12:17:12.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.882+0000 7fb1b84fa640 1 -- 172.21.15.19:0/296740801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1b002d5c0 con 0x7fb1b010c570 2024-04-25T12:17:12.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.883+0000 7fb1a77fe640 1 -- 172.21.15.19:0/296740801 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+23266 (secure 0 0 0) 0x7fb1a809bd10 con 0x7fb1b010c570 2024-04-25T12:17:12.886 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:17:12.886 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":95,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:17:09.419290+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:05:01.658354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:05:02.969078+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/2813829603":"2024-04-26T12:07:09.072296+0000","172.21.15.19:0/1048024792":"2024-04-26T12:07:09.072296+0000","172.21.15.19:6800/2853466745":"2024-04-26T12:07:09.072296+0000","172.21.15.19:0/1094943085":"2024-04-26T12:07:09.072296+0000","172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/2734487323":"2024-04-26T12:07:09.072296+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/2853466745":"2024-04-26T12:07:09.072296+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:17:12.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.885+0000 7fb1b84fa640 1 -- 172.21.15.19:0/296740801 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb19c0a3d20 msgr2=0x7fb19c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:12.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.885+0000 7fb1b84fa640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb19c0a3d20 0x7fb19c0a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009930 tx=0x7fb1ac008040 comp rx=0 tx=0).stop 2024-04-25T12:17:12.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.885+0000 7fb1b84fa640 1 -- 172.21.15.19:0/296740801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1b010c570 msgr2=0x7fb1b0101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:12.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.885+0000 7fb1b84fa640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1b010c570 0x7fb1b0101900 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800b9e0 tx=0x7fb1a800bea0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.885+0000 7fb1b84fa640 1 -- 172.21.15.19:0/296740801 shutdown_connections 2024-04-25T12:17:12.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.885+0000 7fb1b84fa640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb19c0a3d20 0x7fb19c0a61e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.885+0000 7fb1b84fa640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1b010c570 0x7fb1b0101900 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.885+0000 7fb1b84fa640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1b0103b50 0x7fb1b01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.885+0000 7fb1b84fa640 1 --2- 172.21.15.19:0/296740801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1b0103230 0x7fb1b0100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:12.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.885+0000 7fb1b84fa640 1 -- 172.21.15.19:0/296740801 >> 172.21.15.19:0/296740801 conn(0x7fb1b0077fb0 msgr2=0x7fb1b00fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:12.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.885+0000 7fb1b84fa640 1 -- 172.21.15.19:0/296740801 shutdown_connections 2024-04-25T12:17:12.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:12.885+0000 7fb1b84fa640 1 -- 172.21.15.19:0/296740801 wait complete. 2024-04-25T12:17:12.898 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:00:20.898809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:05:01.658354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:05:02.969078+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T12:17:12.899 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:17:12.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:12 smithi169 ceph-mon[32710]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T12:17:12.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:12 smithi150 ceph-mon[32250]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T12:17:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.002+0000 7f6c493db640 1 -- 172.21.15.19:0/3529374005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c4406b1f0 msgr2=0x7f6c44106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.002+0000 7f6c493db640 1 --2- 172.21.15.19:0/3529374005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c4406b1f0 0x7f6c44106b90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009900 tx=0x7f6c2c02a410 comp rx=0 tx=0).stop 2024-04-25T12:17:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.002+0000 7f6c493db640 1 -- 172.21.15.19:0/3529374005 shutdown_connections 2024-04-25T12:17:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.002+0000 7f6c493db640 1 --2- 172.21.15.19:0/3529374005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c44107270 0x7f6c4410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.002+0000 7f6c493db640 1 --2- 172.21.15.19:0/3529374005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c4406b1f0 0x7f6c44106b90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.002+0000 7f6c493db640 1 --2- 172.21.15.19:0/3529374005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c4406a840 0x7f6c4406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.002+0000 7f6c493db640 1 -- 172.21.15.19:0/3529374005 >> 172.21.15.19:0/3529374005 conn(0x7f6c44077eb0 msgr2=0x7f6c440782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.002+0000 7f6c493db640 1 -- 172.21.15.19:0/3529374005 shutdown_connections 2024-04-25T12:17:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.003+0000 7f6c493db640 1 -- 172.21.15.19:0/3529374005 wait complete. 2024-04-25T12:17:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.003+0000 7f6c493db640 1 Processor -- start 2024-04-25T12:17:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.003+0000 7f6c493db640 1 -- start start 2024-04-25T12:17:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.003+0000 7f6c493db640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c4406a840 0x7f6c4419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.003+0000 7f6c493db640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c44107270 0x7f6c4419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.003+0000 7f6c493db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c441986d0 0x7f6c44198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.003+0000 7f6c493db640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44112370 con 0x7f6c441986d0 2024-04-25T12:17:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.003+0000 7f6c493db640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441121f0 con 0x7f6c4406a840 2024-04-25T12:17:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.003+0000 7f6c493db640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441124f0 con 0x7f6c44107270 2024-04-25T12:17:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.003+0000 7f6c437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c441986d0 0x7f6c44198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.003+0000 7f6c437fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c441986d0 0x7f6c44198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37420/0 (socket says 172.21.15.19:37420) 2024-04-25T12:17:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.003+0000 7f6c437fe640 1 -- 172.21.15.19:0/370180062 learned_addr learned my addr 172.21.15.19:0/370180062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.004+0000 7f6c3bfff640 1 --2- 172.21.15.19:0/370180062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c44107270 0x7f6c4419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.004+0000 7f6c42ffd640 1 --2- 172.21.15.19:0/370180062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c4406a840 0x7f6c4419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.004+0000 7f6c437fe640 1 -- 172.21.15.19:0/370180062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c44107270 msgr2=0x7f6c4419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.004+0000 7f6c437fe640 1 --2- 172.21.15.19:0/370180062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c44107270 0x7f6c4419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.004+0000 7f6c437fe640 1 -- 172.21.15.19:0/370180062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c4406a840 msgr2=0x7f6c4419e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.004+0000 7f6c437fe640 1 --2- 172.21.15.19:0/370180062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c4406a840 0x7f6c4419e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.004+0000 7f6c437fe640 1 -- 172.21.15.19:0/370180062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c441992d0 con 0x7f6c441986d0 2024-04-25T12:17:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.004+0000 7f6c437fe640 1 --2- 172.21.15.19:0/370180062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c441986d0 0x7f6c44198b50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00b970 tx=0x7f6c3c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.004+0000 7f6c40ff9640 1 -- 172.21.15.19:0/370180062 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c019070 con 0x7f6c441986d0 2024-04-25T12:17:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.004+0000 7f6c40ff9640 1 -- 172.21.15.19:0/370180062 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f6c3c00cb40 con 0x7f6c441986d0 2024-04-25T12:17:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.004+0000 7f6c493db640 1 -- 172.21.15.19:0/370180062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c441a53b0 con 0x7f6c441986d0 2024-04-25T12:17:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.004+0000 7f6c40ff9640 1 -- 172.21.15.19:0/370180062 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c007650 con 0x7f6c441986d0 2024-04-25T12:17:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.004+0000 7f6c493db640 1 -- 172.21.15.19:0/370180062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c441a5760 con 0x7f6c441986d0 2024-04-25T12:17:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.005+0000 7f6c493db640 1 -- 172.21.15.19:0/370180062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c44103670 con 0x7f6c441986d0 2024-04-25T12:17:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.006+0000 7f6c40ff9640 1 -- 172.21.15.19:0/370180062 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6c3c01c070 con 0x7f6c441986d0 2024-04-25T12:17:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.007+0000 7f6c40ff9640 1 --2- 172.21.15.19:0/370180062 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f6c300a3da0 0x7f6c300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.007+0000 7f6c42ffd640 1 --2- 172.21.15.19:0/370180062 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f6c300a3da0 0x7f6c300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.007+0000 7f6c40ff9640 1 -- 172.21.15.19:0/370180062 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7f6c3c0cfc60 con 0x7f6c441986d0 2024-04-25T12:17:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.007+0000 7f6c42ffd640 1 --2- 172.21.15.19:0/370180062 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f6c300a3da0 0x7f6c300a6260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6c34005fd0 tx=0x7f6c3400b390 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.011+0000 7f6c40ff9640 1 -- 172.21.15.19:0/370180062 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c3c09bd60 con 0x7f6c441986d0 2024-04-25T12:17:13.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.197+0000 7f6c493db640 1 -- 172.21.15.19:0/370180062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6c4410a500 con 0x7f6c441986d0 2024-04-25T12:17:13.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.198+0000 7f6c40ff9640 1 -- 172.21.15.19:0/370180062 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f6c3c0a0060 con 0x7f6c441986d0 2024-04-25T12:17:13.200 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T12:17:13.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.200+0000 7f6c493db640 1 -- 172.21.15.19:0/370180062 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f6c300a3da0 msgr2=0x7f6c300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.200+0000 7f6c493db640 1 --2- 172.21.15.19:0/370180062 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f6c300a3da0 0x7f6c300a6260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6c34005fd0 tx=0x7f6c3400b390 comp rx=0 tx=0).stop 2024-04-25T12:17:13.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.200+0000 7f6c493db640 1 -- 172.21.15.19:0/370180062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c441986d0 msgr2=0x7f6c44198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.200+0000 7f6c493db640 1 --2- 172.21.15.19:0/370180062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c441986d0 0x7f6c44198b50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00b970 tx=0x7f6c3c00be30 comp rx=0 tx=0).stop 2024-04-25T12:17:13.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.200+0000 7f6c493db640 1 -- 172.21.15.19:0/370180062 shutdown_connections 2024-04-25T12:17:13.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.200+0000 7f6c493db640 1 --2- 172.21.15.19:0/370180062 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f6c300a3da0 0x7f6c300a6260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.200+0000 7f6c493db640 1 --2- 172.21.15.19:0/370180062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c441986d0 0x7f6c44198b50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.200+0000 7f6c493db640 1 --2- 172.21.15.19:0/370180062 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c44107270 0x7f6c4419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.200+0000 7f6c493db640 1 --2- 172.21.15.19:0/370180062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c4406a840 0x7f6c4419e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.200+0000 7f6c493db640 1 -- 172.21.15.19:0/370180062 >> 172.21.15.19:0/370180062 conn(0x7f6c44077eb0 msgr2=0x7f6c440fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:13.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.200+0000 7f6c493db640 1 -- 172.21.15.19:0/370180062 shutdown_connections 2024-04-25T12:17:13.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.200+0000 7f6c493db640 1 -- 172.21.15.19:0/370180062 wait complete. 2024-04-25T12:17:13.213 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T12:17:13.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.316+0000 7fc8ee907640 1 -- 172.21.15.19:0/1669436951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8e8106150 msgr2=0x7fc8e810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.316+0000 7fc8ee907640 1 --2- 172.21.15.19:0/1669436951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8e8106150 0x7fc8e810dd20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009ca0 tx=0x7fc8e002a8c0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.316+0000 7fc8ee907640 1 -- 172.21.15.19:0/1669436951 shutdown_connections 2024-04-25T12:17:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.316+0000 7fc8ee907640 1 --2- 172.21.15.19:0/1669436951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8e8106150 0x7fc8e810dd20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.316+0000 7fc8ee907640 1 --2- 172.21.15.19:0/1669436951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e8101a30 0x7fc8e8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.316+0000 7fc8ee907640 1 --2- 172.21.15.19:0/1669436951 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8e8101080 0x7fc8e8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.316+0000 7fc8ee907640 1 -- 172.21.15.19:0/1669436951 >> 172.21.15.19:0/1669436951 conn(0x7fc8e8078000 msgr2=0x7fc8e8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.317+0000 7fc8ee907640 1 -- 172.21.15.19:0/1669436951 shutdown_connections 2024-04-25T12:17:13.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.317+0000 7fc8ee907640 1 -- 172.21.15.19:0/1669436951 wait complete. 2024-04-25T12:17:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.317+0000 7fc8ee907640 1 Processor -- start 2024-04-25T12:17:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.317+0000 7fc8ee907640 1 -- start start 2024-04-25T12:17:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.317+0000 7fc8ee907640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8e8101080 0x7fc8e81987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.317+0000 7fc8ee907640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8e8106150 0x7fc8e8198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.317+0000 7fc8ee907640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e819d020 0x7fc8e819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.317+0000 7fc8ee907640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e81122f0 con 0x7fc8e8101080 2024-04-25T12:17:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.317+0000 7fc8ee907640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e8112170 con 0x7fc8e819d020 2024-04-25T12:17:13.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.317+0000 7fc8ee907640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e8112470 con 0x7fc8e8106150 2024-04-25T12:17:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8e7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8e8101080 0x7fc8e81987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8e7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8e8101080 0x7fc8e81987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37440/0 (socket says 172.21.15.19:37440) 2024-04-25T12:17:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8e7fff640 1 -- 172.21.15.19:0/475584941 learned_addr learned my addr 172.21.15.19:0/475584941 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8ecea4640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e819d020 0x7fc8e819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8e77fe640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8e8106150 0x7fc8e8198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8e7fff640 1 -- 172.21.15.19:0/475584941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8e8106150 msgr2=0x7fc8e8198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8e7fff640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8e8106150 0x7fc8e8198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8e7fff640 1 -- 172.21.15.19:0/475584941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e819d020 msgr2=0x7fc8e819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8e7fff640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e819d020 0x7fc8e819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8e7fff640 1 -- 172.21.15.19:0/475584941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e819db10 con 0x7fc8e8101080 2024-04-25T12:17:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8e77fe640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8e8106150 0x7fc8e8198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:17:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8e7fff640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8e8101080 0x7fc8e81987b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800d950 tx=0x7fc8d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8e57fa640 1 -- 172.21.15.19:0/475584941 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8017070 con 0x7fc8e8101080 2024-04-25T12:17:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8e57fa640 1 -- 172.21.15.19:0/475584941 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fc8d8009c30 con 0x7fc8e8101080 2024-04-25T12:17:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8ee907640 1 -- 172.21.15.19:0/475584941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e819de00 con 0x7fc8e8101080 2024-04-25T12:17:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8e57fa640 1 -- 172.21.15.19:0/475584941 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800f6b0 con 0x7fc8e8101080 2024-04-25T12:17:13.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.318+0000 7fc8ee907640 1 -- 172.21.15.19:0/475584941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e81a55f0 con 0x7fc8e8101080 2024-04-25T12:17:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.319+0000 7fc8ee907640 1 -- 172.21.15.19:0/475584941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e8068d70 con 0x7fc8e8101080 2024-04-25T12:17:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.321+0000 7fc8e57fa640 1 -- 172.21.15.19:0/475584941 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc8d80109c0 con 0x7fc8e8101080 2024-04-25T12:17:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.322+0000 7fc8e57fa640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fc8d00a3d20 0x7fc8d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.322+0000 7fc8e77fe640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fc8d00a3d20 0x7fc8d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.322+0000 7fc8e57fa640 1 -- 172.21.15.19:0/475584941 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7fc8d80d0590 con 0x7fc8e8101080 2024-04-25T12:17:13.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.322+0000 7fc8e77fe640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fc8d00a3d20 0x7fc8d00a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005fd0 tx=0x7fc8dc0094d0 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:13.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.326+0000 7fc8e57fa640 1 -- 172.21.15.19:0/475584941 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8d8009de0 con 0x7fc8e8101080 2024-04-25T12:17:13.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.512+0000 7fc8ee907640 1 -- 172.21.15.19:0/475584941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc8e8101a30 con 0x7fc8e8101080 2024-04-25T12:17:13.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.513+0000 7fc8e57fa640 1 -- 172.21.15.19:0/475584941 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7fc8d809cbd0 con 0x7fc8e8101080 2024-04-25T12:17:13.515 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T12:17:13.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.515+0000 7fc8ee907640 1 -- 172.21.15.19:0/475584941 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fc8d00a3d20 msgr2=0x7fc8d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.515+0000 7fc8ee907640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fc8d00a3d20 0x7fc8d00a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005fd0 tx=0x7fc8dc0094d0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.515+0000 7fc8ee907640 1 -- 172.21.15.19:0/475584941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8e8101080 msgr2=0x7fc8e81987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.515+0000 7fc8ee907640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8e8101080 0x7fc8e81987b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800d950 tx=0x7fc8d800de10 comp rx=0 tx=0).stop 2024-04-25T12:17:13.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.515+0000 7fc8ee907640 1 -- 172.21.15.19:0/475584941 shutdown_connections 2024-04-25T12:17:13.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.515+0000 7fc8ee907640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fc8d00a3d20 0x7fc8d00a61e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.515+0000 7fc8ee907640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e819d020 0x7fc8e819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.515+0000 7fc8ee907640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8e8106150 0x7fc8e8198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.515+0000 7fc8ee907640 1 --2- 172.21.15.19:0/475584941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8e8101080 0x7fc8e81987b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.515+0000 7fc8ee907640 1 -- 172.21.15.19:0/475584941 >> 172.21.15.19:0/475584941 conn(0x7fc8e8078000 msgr2=0x7fc8e80fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:13.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.515+0000 7fc8ee907640 1 -- 172.21.15.19:0/475584941 shutdown_connections 2024-04-25T12:17:13.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.515+0000 7fc8ee907640 1 -- 172.21.15.19:0/475584941 wait complete. 2024-04-25T12:17:13.526 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:13 smithi019 ceph-mon[29213]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:17:13.526 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:13 smithi019 ceph-mon[29213]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:17:13.526 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:13 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/296740801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:17:13.527 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:13 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/370180062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:17:13.528 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T12:17:13.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.631+0000 7f5f36d8e640 1 -- 172.21.15.19:0/1139831582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f30102b90 msgr2=0x7f5f30106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.631+0000 7f5f36d8e640 1 --2- 172.21.15.19:0/1139831582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f30102b90 0x7f5f30106bd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009900 tx=0x7f5f1802a410 comp rx=0 tx=0).stop 2024-04-25T12:17:13.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.631+0000 7f5f36d8e640 1 -- 172.21.15.19:0/1139831582 shutdown_connections 2024-04-25T12:17:13.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.631+0000 7f5f36d8e640 1 --2- 172.21.15.19:0/1139831582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f301072b0 0x7f5f3010ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.631+0000 7f5f36d8e640 1 --2- 172.21.15.19:0/1139831582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f30102b90 0x7f5f30106bd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.631+0000 7f5f36d8e640 1 --2- 172.21.15.19:0/1139831582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f301021e0 0x7f5f301025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.631+0000 7f5f36d8e640 1 -- 172.21.15.19:0/1139831582 >> 172.21.15.19:0/1139831582 conn(0x7f5f300fda60 msgr2=0x7f5f300ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:13.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.631+0000 7f5f36d8e640 1 -- 172.21.15.19:0/1139831582 shutdown_connections 2024-04-25T12:17:13.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.631+0000 7f5f36d8e640 1 -- 172.21.15.19:0/1139831582 wait complete. 2024-04-25T12:17:13.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f36d8e640 1 Processor -- start 2024-04-25T12:17:13.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f36d8e640 1 -- start start 2024-04-25T12:17:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f36d8e640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f301021e0 0x7f5f3019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f36d8e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f301072b0 0x7f5f3019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f36d8e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f30198710 0x7f5f30198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f36d8e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f30112330 con 0x7f5f301072b0 2024-04-25T12:17:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f36d8e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301121b0 con 0x7f5f30198710 2024-04-25T12:17:13.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f36d8e640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5f301124b0 con 0x7f5f301021e0 2024-04-25T12:17:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f301072b0 0x7f5f3019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f301072b0 0x7f5f3019eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37462/0 (socket says 172.21.15.19:37462) 2024-04-25T12:17:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f2ffff640 1 -- 172.21.15.19:0/2071755946 learned_addr learned my addr 172.21.15.19:0/2071755946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f2ffff640 1 -- 172.21.15.19:0/2071755946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f301021e0 msgr2=0x7f5f3019e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f2ffff640 1 --2- 172.21.15.19:0/2071755946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f301021e0 0x7f5f3019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f2ffff640 1 -- 172.21.15.19:0/2071755946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f30198710 msgr2=0x7f5f30198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f2ffff640 1 --2- 172.21.15.19:0/2071755946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f30198710 0x7f5f30198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f2ffff640 1 -- 172.21.15.19:0/2071755946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f30199310 con 0x7f5f301072b0 2024-04-25T12:17:13.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.632+0000 7f5f2ffff640 1 --2- 172.21.15.19:0/2071755946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f301072b0 0x7f5f3019eb90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5f18005940 tx=0x7f5f180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:13.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.633+0000 7f5f2dffb640 1 -- 172.21.15.19:0/2071755946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1803f030 con 0x7f5f301072b0 2024-04-25T12:17:13.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.633+0000 7f5f2dffb640 1 -- 172.21.15.19:0/2071755946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5f180318f0 con 0x7f5f301072b0 2024-04-25T12:17:13.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.633+0000 7f5f36d8e640 1 -- 172.21.15.19:0/2071755946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f301995a0 con 0x7f5f301072b0 2024-04-25T12:17:13.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.633+0000 7f5f2dffb640 1 -- 172.21.15.19:0/2071755946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18035070 con 0x7f5f301072b0 2024-04-25T12:17:13.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.633+0000 7f5f36d8e640 1 -- 172.21.15.19:0/2071755946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f301a5730 con 0x7f5f301072b0 2024-04-25T12:17:13.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.634+0000 7f5f36d8e640 1 -- 172.21.15.19:0/2071755946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f301036b0 con 0x7f5f301072b0 2024-04-25T12:17:13.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.635+0000 7f5f2dffb640 1 -- 172.21.15.19:0/2071755946 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5f18042030 con 0x7f5f301072b0 2024-04-25T12:17:13.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.636+0000 7f5f2dffb640 1 --2- 172.21.15.19:0/2071755946 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f5f1c0a3da0 0x7f5f1c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.636+0000 7f5f34b2a640 1 --2- 172.21.15.19:0/2071755946 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f5f1c0a3da0 0x7f5f1c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:13.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.636+0000 7f5f2dffb640 1 -- 172.21.15.19:0/2071755946 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7f5f180f0ce0 con 0x7f5f301072b0 2024-04-25T12:17:13.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.636+0000 7f5f34b2a640 1 --2- 172.21.15.19:0/2071755946 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f5f1c0a3da0 0x7f5f1c0a6260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f20005fd0 tx=0x7f5f20005c70 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:13.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.640+0000 7f5f2dffb640 1 -- 172.21.15.19:0/2071755946 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f180bd2c0 con 0x7f5f301072b0 2024-04-25T12:17:13.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.829+0000 7f5f36d8e640 1 -- 172.21.15.19:0/2071755946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5f30102b90 con 0x7f5f301072b0 2024-04-25T12:17:13.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.829+0000 7f5f2dffb640 1 -- 172.21.15.19:0/2071755946 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f5f180bcbc0 con 0x7f5f301072b0 2024-04-25T12:17:13.831 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T12:17:13.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.831+0000 7f5f36d8e640 1 -- 172.21.15.19:0/2071755946 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f5f1c0a3da0 msgr2=0x7f5f1c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.831+0000 7f5f36d8e640 1 --2- 172.21.15.19:0/2071755946 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f5f1c0a3da0 0x7f5f1c0a6260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f20005fd0 tx=0x7f5f20005c70 comp rx=0 tx=0).stop 2024-04-25T12:17:13.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.831+0000 7f5f36d8e640 1 -- 172.21.15.19:0/2071755946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f301072b0 msgr2=0x7f5f3019eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.831+0000 7f5f36d8e640 1 --2- 172.21.15.19:0/2071755946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f301072b0 0x7f5f3019eb90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5f18005940 tx=0x7f5f180042f0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.831+0000 7f5f36d8e640 1 -- 172.21.15.19:0/2071755946 shutdown_connections 2024-04-25T12:17:13.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.831+0000 7f5f36d8e640 1 --2- 172.21.15.19:0/2071755946 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f5f1c0a3da0 0x7f5f1c0a6260 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.831+0000 7f5f36d8e640 1 --2- 172.21.15.19:0/2071755946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f30198710 0x7f5f30198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.831+0000 7f5f36d8e640 1 --2- 172.21.15.19:0/2071755946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f301072b0 0x7f5f3019eb90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.831+0000 7f5f36d8e640 1 --2- 172.21.15.19:0/2071755946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f301021e0 0x7f5f3019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.831+0000 7f5f36d8e640 1 -- 172.21.15.19:0/2071755946 >> 172.21.15.19:0/2071755946 conn(0x7f5f300fda60 msgr2=0x7f5f300ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:13.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.832+0000 7f5f36d8e640 1 -- 172.21.15.19:0/2071755946 shutdown_connections 2024-04-25T12:17:13.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.832+0000 7f5f36d8e640 1 -- 172.21.15.19:0/2071755946 wait complete. 2024-04-25T12:17:13.844 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T12:17:13.908 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:13 smithi169 ceph-mon[32710]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:17:13.909 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:13 smithi169 ceph-mon[32710]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:17:13.909 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:13 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/296740801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:17:13.909 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:13 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/370180062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:17:13.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:13 smithi150 ceph-mon[32250]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:17:13.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:13 smithi150 ceph-mon[32250]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T12:17:13.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:13 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/296740801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:17:13.917 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:13 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/370180062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:17:13.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.944+0000 7f50892ae640 1 -- 172.21.15.19:0/268028947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50841010a0 msgr2=0x7f5084101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.944+0000 7f50892ae640 1 --2- 172.21.15.19:0/268028947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50841010a0 0x7f5084101480 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5074009900 tx=0x7f507402a410 comp rx=0 tx=0).stop 2024-04-25T12:17:13.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.944+0000 7f50892ae640 1 -- 172.21.15.19:0/268028947 shutdown_connections 2024-04-25T12:17:13.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.944+0000 7f50892ae640 1 --2- 172.21.15.19:0/268028947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5084106170 0x7f508410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.944+0000 7f50892ae640 1 --2- 172.21.15.19:0/268028947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5084101a50 0x7f5084105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.944+0000 7f50892ae640 1 --2- 172.21.15.19:0/268028947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50841010a0 0x7f5084101480 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.944+0000 7f50892ae640 1 -- 172.21.15.19:0/268028947 >> 172.21.15.19:0/268028947 conn(0x7f5084077f50 msgr2=0x7f5084078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.944+0000 7f50892ae640 1 -- 172.21.15.19:0/268028947 shutdown_connections 2024-04-25T12:17:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.944+0000 7f50892ae640 1 -- 172.21.15.19:0/268028947 wait complete. 2024-04-25T12:17:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.945+0000 7f50892ae640 1 Processor -- start 2024-04-25T12:17:13.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.945+0000 7f50892ae640 1 -- start start 2024-04-25T12:17:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.945+0000 7f50892ae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50841010a0 0x7f5084199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.945+0000 7f50892ae640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5084106170 0x7f5084199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.945+0000 7f50892ae640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f508419a170 0x7f50841a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.945+0000 7f50892ae640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50841111f0 con 0x7f50841010a0 2024-04-25T12:17:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.945+0000 7f50892ae640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5084111070 con 0x7f5084106170 2024-04-25T12:17:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.945+0000 7f50892ae640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f5084111370 con 0x7f508419a170 2024-04-25T12:17:13.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f5082ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50841010a0 0x7f5084199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:13.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f5082ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50841010a0 0x7f5084199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37480/0 (socket says 172.21.15.19:37480) 2024-04-25T12:17:13.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f5082ffd640 1 -- 172.21.15.19:0/3471784275 learned_addr learned my addr 172.21.15.19:0/3471784275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:13.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f50827fc640 1 --2- 172.21.15.19:0/3471784275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5084106170 0x7f5084199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:13.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f50837fe640 1 --2- 172.21.15.19:0/3471784275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f508419a170 0x7f50841a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:13.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f5082ffd640 1 -- 172.21.15.19:0/3471784275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f508419a170 msgr2=0x7f50841a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f5082ffd640 1 --2- 172.21.15.19:0/3471784275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f508419a170 0x7f50841a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f5082ffd640 1 -- 172.21.15.19:0/3471784275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5084106170 msgr2=0x7f5084199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f5082ffd640 1 --2- 172.21.15.19:0/3471784275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5084106170 0x7f5084199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f5082ffd640 1 -- 172.21.15.19:0/3471784275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50841a4ef0 con 0x7f50841010a0 2024-04-25T12:17:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f5082ffd640 1 --2- 172.21.15.19:0/3471784275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50841010a0 0x7f5084199650 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5074006010 tx=0x7f5074004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f506bfff640 1 -- 172.21.15.19:0/3471784275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507403f070 con 0x7f50841010a0 2024-04-25T12:17:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f50892ae640 1 -- 172.21.15.19:0/3471784275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50841a5180 con 0x7f50841010a0 2024-04-25T12:17:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f50892ae640 1 -- 172.21.15.19:0/3471784275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50841a56c0 con 0x7f50841010a0 2024-04-25T12:17:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f506bfff640 1 -- 172.21.15.19:0/3471784275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f5074031e10 con 0x7f50841010a0 2024-04-25T12:17:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.946+0000 7f506bfff640 1 -- 172.21.15.19:0/3471784275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074049060 con 0x7f50841010a0 2024-04-25T12:17:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.947+0000 7f50892ae640 1 -- 172.21.15.19:0/3471784275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5084068d70 con 0x7f50841010a0 2024-04-25T12:17:13.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.948+0000 7f506bfff640 1 -- 172.21.15.19:0/3471784275 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5074042070 con 0x7f50841010a0 2024-04-25T12:17:13.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.949+0000 7f506bfff640 1 --2- 172.21.15.19:0/3471784275 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f506c0a3d20 0x7f506c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:13.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.949+0000 7f506bfff640 1 -- 172.21.15.19:0/3471784275 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7f50740f0fd0 con 0x7f50841010a0 2024-04-25T12:17:13.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.949+0000 7f50827fc640 1 --2- 172.21.15.19:0/3471784275 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f506c0a3d20 0x7f506c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:13.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.949+0000 7f50827fc640 1 --2- 172.21.15.19:0/3471784275 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f506c0a3d20 0x7f506c0a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5084069c90 tx=0x7f507800a4c0 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:13.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:13.953+0000 7f506bfff640 1 -- 172.21.15.19:0/3471784275 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50740bd610 con 0x7f50841010a0 2024-04-25T12:17:14.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:14.192+0000 7f50892ae640 1 -- 172.21.15.19:0/3471784275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f50841093a0 con 0x7f50841010a0 2024-04-25T12:17:14.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:14.200+0000 7f506bfff640 1 -- 172.21.15.19:0/3471784275 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746061 (secure 0 0 0) 0x7f50740bcf10 con 0x7f50841010a0 2024-04-25T12:17:14.213 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:17:14.214 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:17:14.214 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 26, 2024-04-25T12:17:14.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.214 INFO:teuthology.orchestra.run.smithi019.stdout: "active_gid": 15381, 2024-04-25T12:17:14.214 INFO:teuthology.orchestra.run.smithi019.stdout: "active_name": "x", 2024-04-25T12:17:14.214 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addrs": { 2024-04-25T12:17:14.214 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T12:17:14.215 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T12:17:14.215 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-04-25T12:17:14.215 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3657854393 2024-04-25T12:17:14.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.215 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-04-25T12:17:14.216 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-04-25T12:17:14.216 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3657854393 2024-04-25T12:17:14.216 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.216 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T12:17:14.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.216 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addr": "172.21.15.19:6801/3657854393", 2024-04-25T12:17:14.216 INFO:teuthology.orchestra.run.smithi019.stdout: "active_change": "2024-04-25T12:07:09.072518+0000", 2024-04-25T12:17:14.217 INFO:teuthology.orchestra.run.smithi019.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T12:17:14.217 INFO:teuthology.orchestra.run.smithi019.stdout: "available": true, 2024-04-25T12:17:14.217 INFO:teuthology.orchestra.run.smithi019.stdout: "standbys": [ 2024-04-25T12:17:14.217 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.217 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 24959, 2024-04-25T12:17:14.217 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "y", 2024-04-25T12:17:14.217 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-04-25T12:17:14.217 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-04-25T12:17:14.217 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-04-25T12:17:14.218 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.218 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.218 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.218 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:14.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:14.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.219 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:14.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.219 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.220 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.220 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T12:17:14.220 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.220 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.220 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.221 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.221 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.222 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.222 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.222 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.222 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.222 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.222 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.222 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.223 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.223 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.224 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.225 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.225 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.225 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.225 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.226 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.226 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.226 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.226 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.226 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.226 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.226 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.226 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.227 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.227 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.227 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.227 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.227 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.228 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.228 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.228 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.228 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.229 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.229 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.229 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-04-25T12:17:14.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-04-25T12:17:14.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.230 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.230 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.230 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-04-25T12:17:14.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.231 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-04-25T12:17:14.231 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-04-25T12:17:14.231 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-04-25T12:17:14.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-04-25T12:17:14.232 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.233 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-04-25T12:17:14.233 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-04-25T12:17:14.233 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.233 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.234 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.234 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.234 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.234 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.234 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-04-25T12:17:14.234 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.234 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.238 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-04-25T12:17:14.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-04-25T12:17:14.238 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.238 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.238 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.239 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.239 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-04-25T12:17:14.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.239 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.239 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.239 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-04-25T12:17:14.240 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-04-25T12:17:14.240 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.240 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-04-25T12:17:14.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.241 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-04-25T12:17:14.241 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.241 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.241 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.241 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-04-25T12:17:14.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-04-25T12:17:14.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.242 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.242 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.242 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.242 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.242 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-04-25T12:17:14.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.243 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-04-25T12:17:14.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-04-25T12:17:14.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.244 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T12:17:14.244 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.245 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.245 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.245 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-04-25T12:17:14.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-04-25T12:17:14.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.245 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.245 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.246 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.246 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.246 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-04-25T12:17:14.246 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.247 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.247 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.247 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-04-25T12:17:14.247 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.247 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.247 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.248 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-04-25T12:17:14.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-04-25T12:17:14.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.248 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.248 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T12:17:14.249 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.249 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.249 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-04-25T12:17:14.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-04-25T12:17:14.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.250 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.250 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.250 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-04-25T12:17:14.250 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.251 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T12:17:14.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T12:17:14.251 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.251 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.251 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-04-25T12:17:14.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-04-25T12:17:14.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:14.252 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.252 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.252 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:14.252 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T12:17:14.252 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T12:17:14.252 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.253 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T12:17:14.253 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T12:17:14.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.253 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.253 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-04-25T12:17:14.253 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-04-25T12:17:14.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:14.254 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.254 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-04-25T12:17:14.254 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T12:17:14.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-04-25T12:17:14.254 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.254 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T12:17:14.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.255 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.255 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.255 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-04-25T12:17:14.255 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-04-25T12:17:14.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.255 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.256 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.256 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T12:17:14.256 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.256 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.256 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T12:17:14.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T12:17:14.256 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.256 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.257 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-04-25T12:17:14.257 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-04-25T12:17:14.257 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:14.257 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.257 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-04-25T12:17:14.258 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-04-25T12:17:14.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-04-25T12:17:14.258 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.258 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-04-25T12:17:14.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T12:17:14.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.259 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-04-25T12:17:14.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-04-25T12:17:14.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.259 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-04-25T12:17:14.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.260 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T12:17:14.260 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T12:17:14.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.260 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-04-25T12:17:14.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-04-25T12:17:14.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:14.261 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:14.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T12:17:14.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T12:17:14.261 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.261 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T12:17:14.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T12:17:14.262 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.262 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.262 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.263 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.263 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.263 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.263 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.263 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.263 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.263 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.263 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.263 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.264 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.264 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.264 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.264 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.264 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.264 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.264 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.267 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.267 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.267 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.267 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.267 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.267 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.268 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.268 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.268 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.268 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.268 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.268 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.269 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.269 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.269 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.269 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.269 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.269 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.270 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.270 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.270 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.270 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.271 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-04-25T12:17:14.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-04-25T12:17:14.272 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:14.272 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.272 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.272 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:14.272 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.272 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.273 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T12:17:14.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.273 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.273 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.273 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T12:17:14.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T12:17:14.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.274 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-04-25T12:17:14.274 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.274 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-04-25T12:17:14.274 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-04-25T12:17:14.275 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-04-25T12:17:14.275 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-04-25T12:17:14.275 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-04-25T12:17:14.275 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.275 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-04-25T12:17:14.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.275 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.275 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.276 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-04-25T12:17:14.276 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-04-25T12:17:14.276 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.276 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.276 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.276 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.276 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.277 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.277 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T12:17:14.277 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.277 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.277 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.277 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:14.278 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:14.278 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.278 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.278 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.278 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:14.278 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.278 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.279 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T12:17:14.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.279 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.279 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-04-25T12:17:14.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-04-25T12:17:14.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.279 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T12:17:14.280 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T12:17:14.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.280 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T12:17:14.280 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T12:17:14.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.281 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.281 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-04-25T12:17:14.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-04-25T12:17:14.281 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:14.281 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.281 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:14.282 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T12:17:14.282 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.282 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.282 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.283 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.283 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-04-25T12:17:14.283 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.283 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.283 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.283 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-04-25T12:17:14.284 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-04-25T12:17:14.284 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:14.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T12:17:14.284 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.284 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.284 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.284 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T12:17:14.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.285 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-04-25T12:17:14.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-04-25T12:17:14.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.285 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-04-25T12:17:14.286 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.286 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.286 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T12:17:14.286 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.286 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.287 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-04-25T12:17:14.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-04-25T12:17:14.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.287 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.287 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-04-25T12:17:14.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.288 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.288 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T12:17:14.288 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.289 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.289 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-04-25T12:17:14.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-04-25T12:17:14.289 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.289 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.289 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.290 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.290 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.290 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T12:17:14.290 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T12:17:14.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.291 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-04-25T12:17:14.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-04-25T12:17:14.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.291 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.291 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:14.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.291 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.292 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T12:17:14.292 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.292 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.292 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.292 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-04-25T12:17:14.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T12:17:14.293 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:14.293 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.293 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-04-25T12:17:14.293 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T12:17:14.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.294 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.294 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.294 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-04-25T12:17:14.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-04-25T12:17:14.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.295 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.295 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.295 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.295 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-04-25T12:17:14.295 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-04-25T12:17:14.295 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-04-25T12:17:14.295 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.296 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T12:17:14.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.296 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.296 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.296 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-04-25T12:17:14.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-04-25T12:17:14.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.297 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.297 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.297 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T12:17:14.297 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.297 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.298 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-04-25T12:17:14.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-04-25T12:17:14.298 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.298 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.298 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.298 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.299 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T12:17:14.299 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.299 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.300 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-04-25T12:17:14.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-04-25T12:17:14.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.300 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.300 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.300 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.300 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.300 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T12:17:14.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.301 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.301 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-04-25T12:17:14.301 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-04-25T12:17:14.301 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.302 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.302 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T12:17:14.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.302 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.302 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T12:17:14.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.303 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-04-25T12:17:14.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-04-25T12:17:14.303 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.303 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T12:17:14.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.304 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.304 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.304 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-04-25T12:17:14.304 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.304 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.304 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.305 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-04-25T12:17:14.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-04-25T12:17:14.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.305 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.305 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.305 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T12:17:14.305 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.305 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-04-25T12:17:14.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.306 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-04-25T12:17:14.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-04-25T12:17:14.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T12:17:14.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.307 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T12:17:14.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.308 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.308 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.308 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-04-25T12:17:14.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-04-25T12:17:14.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T12:17:14.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-04-25T12:17:14.309 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.310 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-04-25T12:17:14.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-04-25T12:17:14.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.310 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T12:17:14.310 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.311 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-04-25T12:17:14.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.311 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.311 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-04-25T12:17:14.312 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-04-25T12:17:14.312 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.312 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.312 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T12:17:14.312 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.312 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.313 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-04-25T12:17:14.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.313 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.313 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-04-25T12:17:14.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-04-25T12:17:14.313 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T12:17:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-04-25T12:17:14.314 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.315 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.315 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.315 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-04-25T12:17:14.315 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-04-25T12:17:14.315 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.315 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.315 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T12:17:14.316 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.316 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.316 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.316 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-04-25T12:17:14.316 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.316 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.316 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.317 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-04-25T12:17:14.317 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-04-25T12:17:14.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.317 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.317 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.317 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T12:17:14.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.318 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.318 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-04-25T12:17:14.318 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.318 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.318 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.318 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-04-25T12:17:14.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-04-25T12:17:14.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.319 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.319 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T12:17:14.319 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.319 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.319 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T12:17:14.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.320 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-04-25T12:17:14.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-04-25T12:17:14.320 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T12:17:14.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.321 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.321 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-04-25T12:17:14.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.322 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-04-25T12:17:14.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-04-25T12:17:14.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.322 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T12:17:14.322 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.323 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T12:17:14.323 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.323 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-04-25T12:17:14.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-04-25T12:17:14.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.324 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.324 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T12:17:14.324 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.324 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.324 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.325 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-04-25T12:17:14.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.325 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.325 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.325 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-04-25T12:17:14.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-04-25T12:17:14.325 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.325 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.326 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.326 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T12:17:14.326 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.326 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.326 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-04-25T12:17:14.326 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.326 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.327 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.327 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-04-25T12:17:14.327 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-04-25T12:17:14.327 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.327 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T12:17:14.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.328 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.328 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.328 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-04-25T12:17:14.328 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.328 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.328 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.328 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-04-25T12:17:14.329 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-04-25T12:17:14.329 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.329 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.329 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.329 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.329 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.329 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.330 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.330 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T12:17:14.330 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.330 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.330 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.330 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-04-25T12:17:14.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-04-25T12:17:14.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.331 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.331 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.331 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:14.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.331 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.331 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.332 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T12:17:14.332 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.332 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-04-25T12:17:14.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T12:17:14.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.333 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.333 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-04-25T12:17:14.333 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.333 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.333 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T12:17:14.333 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.334 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-04-25T12:17:14.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-04-25T12:17:14.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.334 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-04-25T12:17:14.335 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T12:17:14.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.335 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.336 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-04-25T12:17:14.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-04-25T12:17:14.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.336 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.336 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-04-25T12:17:14.336 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.336 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-04-25T12:17:14.337 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.337 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.337 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.337 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-04-25T12:17:14.337 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-04-25T12:17:14.338 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.338 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.338 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.338 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.338 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.338 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.338 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T12:17:14.338 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.339 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.339 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-04-25T12:17:14.339 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-04-25T12:17:14.339 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.339 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:14.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.340 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.340 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.340 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-04-25T12:17:14.340 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.340 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.341 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.341 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-04-25T12:17:14.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-04-25T12:17:14.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.341 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.341 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T12:17:14.342 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.342 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.342 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T12:17:14.342 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.342 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.343 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-04-25T12:17:14.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-04-25T12:17:14.343 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.343 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:14.344 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.344 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-04-25T12:17:14.344 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.344 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.344 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.345 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-04-25T12:17:14.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-04-25T12:17:14.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.345 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.346 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.346 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T12:17:14.346 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.346 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.346 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.347 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-04-25T12:17:14.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-04-25T12:17:14.347 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.347 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.347 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.347 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.347 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.347 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.348 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T12:17:14.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.348 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.348 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.349 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.349 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.350 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.350 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.350 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.350 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.350 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.350 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.351 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.351 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-04-25T12:17:14.352 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-04-25T12:17:14.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.352 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.353 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.353 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T12:17:14.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.353 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.354 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.354 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.355 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T12:17:14.355 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.355 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.355 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.355 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.355 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.356 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.356 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.356 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.356 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.356 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.356 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.357 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.357 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.357 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.357 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.357 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.358 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.358 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.359 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T12:17:14.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T12:17:14.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.360 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.360 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.360 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.360 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T12:17:14.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.361 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.361 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.361 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T12:17:14.361 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T12:17:14.361 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.362 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-04-25T12:17:14.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.362 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T12:17:14.362 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.363 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-04-25T12:17:14.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-04-25T12:17:14.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.364 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:14.364 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.364 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-04-25T12:17:14.364 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.364 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.365 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-04-25T12:17:14.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-04-25T12:17:14.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.365 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.365 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:14.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.365 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T12:17:14.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.366 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-04-25T12:17:14.366 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-04-25T12:17:14.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.367 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.367 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.368 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-04-25T12:17:14.368 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.368 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.368 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.368 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T12:17:14.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T12:17:14.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-04-25T12:17:14.369 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.369 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.369 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.369 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-04-25T12:17:14.370 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-04-25T12:17:14.370 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T12:17:14.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.370 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.370 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.370 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-04-25T12:17:14.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-04-25T12:17:14.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-04-25T12:17:14.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.371 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T12:17:14.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.372 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-04-25T12:17:14.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-04-25T12:17:14.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.373 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.373 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.373 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T12:17:14.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.373 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T12:17:14.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.374 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.374 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.374 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-04-25T12:17:14.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-04-25T12:17:14.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.375 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.375 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.375 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.375 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T12:17:14.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.376 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.376 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-04-25T12:17:14.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-04-25T12:17:14.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.377 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T12:17:14.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.378 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-04-25T12:17:14.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-04-25T12:17:14.378 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.378 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.378 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.378 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.378 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.379 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.379 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.379 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T12:17:14.379 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.379 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.379 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.380 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-04-25T12:17:14.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-04-25T12:17:14.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.381 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T12:17:14.381 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.381 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.382 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-04-25T12:17:14.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-04-25T12:17:14.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.382 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.382 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.382 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T12:17:14.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.383 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-04-25T12:17:14.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-04-25T12:17:14.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-04-25T12:17:14.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.384 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.384 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-04-25T12:17:14.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.385 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-04-25T12:17:14.385 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-04-25T12:17:14.385 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.385 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.386 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.386 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.386 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T12:17:14.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.386 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.387 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-04-25T12:17:14.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-04-25T12:17:14.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.387 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.388 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T12:17:14.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.389 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.389 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-04-25T12:17:14.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-04-25T12:17:14.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.390 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.390 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T12:17:14.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.391 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.391 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-04-25T12:17:14.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-04-25T12:17:14.391 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.392 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T12:17:14.392 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.392 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.392 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.392 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-04-25T12:17:14.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-04-25T12:17:14.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.393 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.393 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.393 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.393 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.393 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.394 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T12:17:14.394 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.394 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.394 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.394 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-04-25T12:17:14.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-04-25T12:17:14.395 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.395 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.395 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T12:17:14.396 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.396 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.396 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.396 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.396 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-04-25T12:17:14.397 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.397 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.397 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.397 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.398 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.399 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.399 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.399 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.399 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.399 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.399 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.400 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.400 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.400 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.400 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.401 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.401 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.402 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.402 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.403 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.403 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.403 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.403 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.403 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.404 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.404 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.404 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.404 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.404 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.404 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.404 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.405 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.405 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.406 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.406 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.406 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.406 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.406 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.407 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-04-25T12:17:14.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-04-25T12:17:14.407 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.407 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-04-25T12:17:14.408 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.408 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.408 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T12:17:14.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.408 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.409 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-04-25T12:17:14.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-04-25T12:17:14.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-04-25T12:17:14.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.410 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T12:17:14.411 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.411 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.411 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.411 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-04-25T12:17:14.412 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.412 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.412 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.412 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T12:17:14.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T12:17:14.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:14.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.414 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T12:17:14.414 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T12:17:14.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.415 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.415 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.415 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.415 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.416 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T12:17:14.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T12:17:14.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.416 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.416 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.417 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.417 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.417 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.417 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-04-25T12:17:14.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T12:17:14.418 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.418 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.418 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.418 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.418 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.418 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.419 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.419 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T12:17:14.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T12:17:14.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.420 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.420 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.421 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.421 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.421 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.421 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T12:17:14.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T12:17:14.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.422 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.422 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.422 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.422 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.422 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.423 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.423 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.423 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.423 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T12:17:14.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T12:17:14.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.424 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.424 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.424 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.424 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.424 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.425 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.425 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.425 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T12:17:14.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T12:17:14.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T12:17:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T12:17:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T12:17:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T12:17:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T12:17:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T12:17:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T12:17:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T12:17:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T12:17:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T12:17:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T12:17:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T12:17:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T12:17:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T12:17:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T12:17:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T12:17:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T12:17:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-04-25T12:17:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-04-25T12:17:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T12:17:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T12:17:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T12:17:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T12:17:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T12:17:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T12:17:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T12:17:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T12:17:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T12:17:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T12:17:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T12:17:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T12:17:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T12:17:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-04-25T12:17:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T12:17:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T12:17:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-04-25T12:17:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T12:17:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T12:17:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.458 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.458 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T12:17:14.458 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T12:17:14.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.458 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.458 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.458 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T12:17:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T12:17:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T12:17:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T12:17:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T12:17:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T12:17:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T12:17:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T12:17:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T12:17:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T12:17:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T12:17:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T12:17:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T12:17:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T12:17:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T12:17:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T12:17:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T12:17:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T12:17:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T12:17:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T12:17:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T12:17:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T12:17:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T12:17:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-04-25T12:17:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T12:17:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T12:17:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-04-25T12:17:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T12:17:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T12:17:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T12:17:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T12:17:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T12:17:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T12:17:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T12:17:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T12:17:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T12:17:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T12:17:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T12:17:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T12:17:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T12:17:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T12:17:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T12:17:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T12:17:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T12:17:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T12:17:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-04-25T12:17:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-04-25T12:17:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-04-25T12:17:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-04-25T12:17:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-04-25T12:17:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-04-25T12:17:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-04-25T12:17:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-04-25T12:17:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-04-25T12:17:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-04-25T12:17:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.501 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T12:17:14.501 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T12:17:14.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.501 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.501 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.501 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.502 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.502 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.502 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.502 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.502 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.503 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.503 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.503 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.503 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.503 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.503 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.503 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.504 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.504 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.504 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.504 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.504 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.504 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.505 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.505 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.505 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.505 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.505 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.505 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.506 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.506 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.506 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.506 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.506 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.506 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.506 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.508 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.508 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.508 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.508 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.508 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.508 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.509 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.509 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.509 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.509 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.509 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.509 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.509 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.509 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.510 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.510 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.510 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.510 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.510 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.510 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.510 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.510 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.511 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.511 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.511 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-04-25T12:17:14.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-04-25T12:17:14.511 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.512 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.512 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.512 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.512 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.512 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.512 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-04-25T12:17:14.512 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.513 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.513 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-04-25T12:17:14.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T12:17:14.513 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.513 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.513 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.514 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.514 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.514 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.514 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.514 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.514 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.514 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.515 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T12:17:14.515 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T12:17:14.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.515 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.515 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T12:17:14.515 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.515 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.516 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.516 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.516 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.516 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T12:17:14.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T12:17:14.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.517 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-04-25T12:17:14.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.517 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.518 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.519 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T12:17:14.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T12:17:14.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.519 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.520 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.520 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.520 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.520 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.520 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.521 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.521 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-04-25T12:17:14.521 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-04-25T12:17:14.521 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.521 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.522 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.522 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-04-25T12:17:14.522 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.522 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.522 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.522 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.522 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.522 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.523 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.523 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T12:17:14.523 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T12:17:14.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.523 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.523 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.523 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-04-25T12:17:14.524 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.524 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.524 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.524 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.524 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-04-25T12:17:14.524 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.524 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.524 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.525 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.525 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.525 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T12:17:14.525 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T12:17:14.525 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.525 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T12:17:14.526 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T12:17:14.526 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T12:17:14.526 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.526 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.526 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.526 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.526 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.527 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-04-25T12:17:14.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-04-25T12:17:14.527 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.527 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.527 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.527 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.527 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.528 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.528 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.528 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.528 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.528 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.528 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.528 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.528 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-25T12:17:14.529 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.529 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.529 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.529 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-04-25T12:17:14.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-04-25T12:17:14.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.529 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.530 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.530 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.530 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.530 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.530 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-04-25T12:17:14.530 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.530 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.531 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.531 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.531 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.531 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.531 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.532 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.532 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.532 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.532 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.532 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.532 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.532 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.532 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.532 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.533 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.533 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.533 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.533 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.533 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.534 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.534 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.534 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.534 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.534 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.534 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.534 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.535 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.535 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.535 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.535 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.535 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.535 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.535 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.536 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.536 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.536 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.536 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.536 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.536 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.536 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.537 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.537 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.537 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.537 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.537 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.538 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.538 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.538 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.538 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.538 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.538 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.538 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.538 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.538 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.539 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-04-25T12:17:14.539 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-04-25T12:17:14.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.539 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.539 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-04-25T12:17:14.539 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.539 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.540 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.540 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T12:17:14.540 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.540 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.540 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.540 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-04-25T12:17:14.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-04-25T12:17:14.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.541 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-04-25T12:17:14.541 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.541 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.541 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T12:17:14.541 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.542 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.542 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.542 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-04-25T12:17:14.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-04-25T12:17:14.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.542 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.542 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.543 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-04-25T12:17:14.543 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.543 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-04-25T12:17:14.543 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.543 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.544 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-04-25T12:17:14.544 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-04-25T12:17:14.544 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.544 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.544 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.544 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T12:17:14.544 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.544 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.545 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T12:17:14.545 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.545 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.545 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.545 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-04-25T12:17:14.545 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-04-25T12:17:14.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.546 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.546 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.546 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.546 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.546 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T12:17:14.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.547 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.547 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.547 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:14.547 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:14.547 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.547 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:14.548 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.548 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.548 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.548 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T12:17:14.548 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.548 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.549 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.549 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-04-25T12:17:14.549 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-04-25T12:17:14.549 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.549 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.549 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.549 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-04-25T12:17:14.549 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.550 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.550 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.550 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T12:17:14.550 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.550 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.550 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.550 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.550 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.551 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.551 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-04-25T12:17:14.551 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.551 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.551 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.551 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.551 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.552 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.552 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.552 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.552 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.552 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.552 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.552 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.552 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.553 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.553 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.553 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.553 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.553 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.553 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.553 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.553 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.554 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.554 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.554 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.554 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.554 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.555 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.555 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.555 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.555 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.555 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.555 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.555 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.555 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.556 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.556 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.556 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.556 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.556 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.556 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.556 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.557 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.557 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.557 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.557 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.557 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.557 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.557 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.557 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.558 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.558 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.558 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.558 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.558 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.558 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.559 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.559 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.559 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.559 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.559 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.559 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.559 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.559 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.559 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.560 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.560 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.560 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-04-25T12:17:14.560 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-04-25T12:17:14.560 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.560 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T12:17:14.561 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.561 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.561 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.561 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.561 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.561 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.562 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.562 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-04-25T12:17:14.562 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-04-25T12:17:14.562 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.562 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.562 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.562 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-04-25T12:17:14.563 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.563 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.563 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.563 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.563 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.563 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.563 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.564 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:14.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:14.564 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.564 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.564 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:14.564 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.564 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.565 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.565 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.565 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.565 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.565 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.565 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.565 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.565 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-04-25T12:17:14.566 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-04-25T12:17:14.566 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-04-25T12:17:14.566 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.566 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-04-25T12:17:14.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-04-25T12:17:14.566 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.566 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.567 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.567 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-04-25T12:17:14.567 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.567 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.567 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T12:17:14.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.568 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-04-25T12:17:14.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-04-25T12:17:14.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.568 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-04-25T12:17:14.568 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.569 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T12:17:14.569 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.569 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.569 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.569 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-04-25T12:17:14.569 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-04-25T12:17:14.569 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.570 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.570 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.570 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.570 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.570 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.570 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-04-25T12:17:14.570 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.571 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.571 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.571 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:14.571 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:14.571 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.571 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.571 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.571 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-04-25T12:17:14.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-04-25T12:17:14.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.572 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T12:17:14.572 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.572 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.572 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.572 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.573 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.573 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.573 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.574 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.574 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.574 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.574 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.574 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.574 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.574 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.575 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.575 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.575 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.575 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.575 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.575 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.576 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.576 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.576 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.576 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.576 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.577 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.577 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.577 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.578 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.578 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.578 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.578 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.578 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.578 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.578 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.578 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.578 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.579 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.579 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.579 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.579 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.579 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.579 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.579 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.579 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.580 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.580 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.580 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.580 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.580 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-04-25T12:17:14.580 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-04-25T12:17:14.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.581 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.581 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.581 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.581 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-04-25T12:17:14.581 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.582 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.582 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.582 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-04-25T12:17:14.582 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-04-25T12:17:14.582 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.582 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.582 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.582 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-04-25T12:17:14.583 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.583 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.583 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-04-25T12:17:14.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.583 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.583 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.583 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T12:17:14.584 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T12:17:14.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.584 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.584 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-04-25T12:17:14.584 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.584 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.584 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.585 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T12:17:14.585 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.585 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-04-25T12:17:14.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-04-25T12:17:14.586 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.586 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.586 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.586 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T12:17:14.586 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T12:17:14.586 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-04-25T12:17:14.586 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.586 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T12:17:14.586 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.587 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.587 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.587 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.587 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-04-25T12:17:14.587 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-04-25T12:17:14.587 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.587 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.587 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.587 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.588 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.588 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.588 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.588 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-04-25T12:17:14.588 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.588 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.588 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.588 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.589 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-04-25T12:17:14.589 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-04-25T12:17:14.589 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.589 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.589 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-04-25T12:17:14.589 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.589 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.589 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.590 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T12:17:14.590 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.590 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.590 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.590 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.590 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.590 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.591 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-04-25T12:17:14.591 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.591 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.591 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.591 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.591 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.591 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.591 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.591 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.592 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.592 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.592 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.592 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.592 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.592 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.592 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.592 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.592 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.592 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.593 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.593 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.593 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.593 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.593 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.593 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.593 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.593 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.594 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.594 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.594 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.594 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.594 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.594 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.594 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.594 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.595 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.595 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.595 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.595 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.595 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.595 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.595 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.596 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.596 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.596 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.596 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.596 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.596 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.596 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.596 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.597 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.597 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.597 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.597 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.597 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.597 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.597 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.597 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.597 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.598 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.598 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.598 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.598 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.598 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.598 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.598 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.598 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.599 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.599 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.599 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.599 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.599 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.599 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.599 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.599 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-04-25T12:17:14.600 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.600 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.600 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.600 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.600 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.600 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.600 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.601 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.601 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.601 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.601 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.601 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.601 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.601 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.601 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.601 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.602 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.602 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.602 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.602 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.602 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.602 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.602 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.603 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.603 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.603 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.603 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.603 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.604 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.604 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.604 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.604 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.604 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.604 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.604 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.604 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.605 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.605 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.605 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.605 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.605 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.605 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.605 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.605 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.606 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.606 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.606 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.606 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.606 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.606 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.606 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.606 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.607 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.607 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.607 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.607 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.607 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.607 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.608 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.608 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.608 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.608 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.608 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.608 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.608 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.608 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-04-25T12:17:14.608 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.609 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.609 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.609 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-04-25T12:17:14.609 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-04-25T12:17:14.609 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.609 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.609 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.609 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-04-25T12:17:14.609 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.610 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.610 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.610 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T12:17:14.610 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.610 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.610 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.610 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.610 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-04-25T12:17:14.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-04-25T12:17:14.611 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.611 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.611 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.611 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:14.611 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-04-25T12:17:14.611 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.611 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.611 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T12:17:14.612 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.612 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.612 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.612 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.612 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.612 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.612 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.612 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.612 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.613 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.613 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.613 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.613 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.613 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.613 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.613 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.613 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.614 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.615 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.615 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.615 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.615 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.615 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.615 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.615 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.615 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.615 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.616 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.616 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.616 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.616 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.616 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.616 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.616 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.617 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.617 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.617 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.617 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.617 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.617 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.617 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.617 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.618 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.618 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.618 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.618 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.618 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.618 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.618 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.619 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.619 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.619 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.619 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.619 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.619 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.619 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.620 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.620 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.620 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.620 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.620 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.620 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.620 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.620 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.621 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.621 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.621 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.621 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.621 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.621 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.621 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-04-25T12:17:14.621 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.621 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.622 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.622 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-04-25T12:17:14.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-04-25T12:17:14.622 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.622 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.622 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.622 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-04-25T12:17:14.622 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.623 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.623 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.623 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-04-25T12:17:14.623 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T12:17:14.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.623 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.623 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.623 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.624 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.624 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.624 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.624 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.624 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.624 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.624 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.624 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.625 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.625 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.625 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.625 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.625 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.625 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.625 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.625 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.626 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.626 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.626 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.626 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.626 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.626 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.627 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.627 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.627 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.627 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.627 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.627 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.627 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.627 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.628 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.628 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.628 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.628 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.628 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.628 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.629 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.629 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.629 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.629 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.629 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.629 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.629 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.629 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.630 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.630 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.630 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.630 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.630 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.630 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.630 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.631 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.631 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.631 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.631 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.631 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.631 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.631 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.631 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-04-25T12:17:14.632 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-04-25T12:17:14.632 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.632 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.632 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.632 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.632 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.632 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.632 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-04-25T12:17:14.633 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T12:17:14.633 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.633 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.633 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-04-25T12:17:14.633 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-04-25T12:17:14.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.633 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.634 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-04-25T12:17:14.634 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.634 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.634 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.634 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-04-25T12:17:14.634 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.634 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.634 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.635 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-04-25T12:17:14.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-04-25T12:17:14.635 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.635 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.635 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.635 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-04-25T12:17:14.635 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.635 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.636 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.636 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T12:17:14.636 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.636 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.636 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.636 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-04-25T12:17:14.636 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-04-25T12:17:14.636 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.637 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.637 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.637 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.637 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.637 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.637 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-04-25T12:17:14.637 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.638 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.638 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.638 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-04-25T12:17:14.638 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-04-25T12:17:14.638 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.638 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.639 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-04-25T12:17:14.639 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.639 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.639 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.639 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T12:17:14.639 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T12:17:14.640 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.640 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.640 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.640 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.640 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.640 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-04-25T12:17:14.640 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.640 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.641 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.641 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.641 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.641 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.641 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.641 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.641 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.641 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.642 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.642 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.642 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.642 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.642 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.642 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.642 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.642 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.643 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.643 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.643 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.643 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.643 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.644 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.644 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.644 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.644 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.644 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.644 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.644 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.644 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.644 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.645 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.645 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.645 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.645 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.645 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.646 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.646 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.646 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.646 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.646 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.646 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.646 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.646 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.647 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.647 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.647 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.647 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.647 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.647 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.647 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.647 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.648 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.648 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.648 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.648 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.648 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.648 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.649 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.649 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.649 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.649 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.649 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.649 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.649 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.650 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.650 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-04-25T12:17:14.650 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.650 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.650 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.650 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.650 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.650 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.651 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.651 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.651 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.651 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.651 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.651 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.651 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.651 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.651 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.652 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.652 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.652 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.652 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.652 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.652 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.652 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.653 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.653 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.653 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.653 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.653 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.653 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.654 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.654 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.654 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.654 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.654 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.654 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.655 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.655 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.655 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.655 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.655 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.656 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.656 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.656 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.656 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.656 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.656 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.656 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.656 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.656 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.657 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.657 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.657 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.657 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.657 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.657 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.657 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.658 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.658 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.658 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.658 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.658 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.658 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.658 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.658 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.659 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.659 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.659 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.659 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.659 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.659 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-04-25T12:17:14.659 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.659 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.660 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.660 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.660 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.660 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.660 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.660 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.660 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.660 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.660 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.661 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.661 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.661 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.661 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.661 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.661 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.661 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.661 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.662 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.662 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.662 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.662 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.662 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.662 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.662 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.663 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.663 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.663 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.663 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.663 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.663 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.664 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.664 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.664 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.664 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.664 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.664 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.664 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.665 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.665 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.665 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.665 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.665 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.665 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.665 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.665 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.665 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.666 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.666 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.666 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.666 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.666 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.666 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.667 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.667 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.667 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.667 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.667 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.667 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.667 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.667 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.667 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.668 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.668 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.668 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.668 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.668 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.668 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.669 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T12:17:14.669 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.669 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.669 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.669 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-04-25T12:17:14.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-04-25T12:17:14.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.670 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.670 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.670 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.670 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.670 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T12:17:14.670 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.671 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.671 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.671 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.671 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.671 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.671 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.671 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.672 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.672 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.672 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.672 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.672 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.672 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.672 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.672 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.673 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.673 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.673 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.673 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.673 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.673 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.673 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.673 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.674 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.674 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.674 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.674 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.674 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.674 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.674 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.674 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.675 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.675 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.675 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.675 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.675 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.676 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.676 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.676 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.676 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.676 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.676 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.677 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.677 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.677 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.677 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.677 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.677 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.677 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.677 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.678 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.678 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.678 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.678 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.678 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.678 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.679 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.679 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.679 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.679 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.679 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.679 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.679 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-04-25T12:17:14.679 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T12:17:14.680 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.680 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.680 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.680 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.680 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.680 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.680 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.680 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-25T12:17:14.680 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-04-25T12:17:14.681 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-04-25T12:17:14.681 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-04-25T12:17:14.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.681 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.681 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.681 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.681 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.682 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-04-25T12:17:14.682 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.682 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.682 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.682 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.682 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.682 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.683 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.683 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.683 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.683 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.683 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.683 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.683 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.683 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.683 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.684 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.684 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.684 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.684 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.684 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.684 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.684 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.685 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.685 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.685 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.685 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.685 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.685 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.685 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.686 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.686 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.686 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.686 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.686 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.686 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.686 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.687 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.687 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.687 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.687 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.687 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.687 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.687 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.687 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.687 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.688 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.688 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.688 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.688 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.688 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.688 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.688 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.688 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.689 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.689 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.689 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.689 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.689 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.690 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.690 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.690 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.690 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.690 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.690 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.690 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.690 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.691 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.691 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-04-25T12:17:14.691 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.691 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.691 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.691 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.692 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.692 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.692 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.692 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.692 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.692 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.692 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.693 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.693 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.693 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.693 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.693 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.693 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.693 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.693 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.693 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.694 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.694 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.694 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.694 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.694 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.694 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.695 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.695 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.695 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.695 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.695 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.695 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.695 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.695 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.696 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.696 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.696 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.696 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.696 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.696 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.696 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.697 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.697 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.697 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.697 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.697 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.697 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.697 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.697 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.698 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.698 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.698 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.698 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.698 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.699 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.699 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.699 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.699 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.699 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.699 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.699 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.700 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.700 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.700 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.700 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.700 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-04-25T12:17:14.701 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.701 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.701 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.701 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.701 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.701 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.701 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.702 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.702 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.702 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.702 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.702 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.702 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.702 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.702 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.702 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.703 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.703 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.703 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.703 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.703 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.703 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.703 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.704 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.704 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.704 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.704 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.704 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.704 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.704 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.704 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.705 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.705 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.705 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.705 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.705 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.705 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.705 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.706 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.706 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.706 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.706 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.706 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.706 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.706 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.706 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.707 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.707 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.707 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.707 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.707 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.707 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.707 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.708 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.708 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.708 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.708 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.708 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.708 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.709 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.709 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.709 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.709 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.709 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.709 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.709 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.709 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.710 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:14.710 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:14.710 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.710 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.710 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.710 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:14.710 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.710 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.711 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.711 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.711 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.711 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.711 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.711 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-04-25T12:17:14.711 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-04-25T12:17:14.712 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:14.712 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.712 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T12:17:14.712 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-04-25T12:17:14.712 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.712 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.712 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-04-25T12:17:14.713 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T12:17:14.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.713 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.713 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.713 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.713 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-04-25T12:17:14.714 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.714 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.714 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.714 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-04-25T12:17:14.714 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-04-25T12:17:14.714 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.714 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.715 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.715 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.715 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.715 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T12:17:14.715 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.715 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.716 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T12:17:14.716 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T12:17:14.716 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.716 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.716 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.716 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.716 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.717 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.717 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.717 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.717 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.717 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.717 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.718 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.718 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.718 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.718 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.718 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.718 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.718 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.719 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.719 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.719 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.719 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.719 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.719 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.719 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.719 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.719 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.720 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.720 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.720 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.720 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.720 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.720 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.720 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.721 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.721 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.721 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.721 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.721 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.721 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.721 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.721 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.722 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.722 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.722 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.722 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.722 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.722 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.722 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.723 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.723 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.723 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.723 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.723 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.723 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.723 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.724 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.724 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.724 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.724 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.724 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.724 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.725 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.725 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.725 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.725 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.725 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.725 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.725 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.726 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.726 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.726 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.726 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-04-25T12:17:14.726 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-04-25T12:17:14.726 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.726 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.727 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.727 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-04-25T12:17:14.727 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.727 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.727 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.727 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-04-25T12:17:14.727 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.727 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.728 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.728 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:14.728 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:14.728 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.728 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.728 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.728 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T12:17:14.728 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.729 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.729 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.729 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-04-25T12:17:14.729 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.729 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.729 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.729 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.729 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.730 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.730 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-04-25T12:17:14.730 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.730 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.730 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.730 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-04-25T12:17:14.730 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-04-25T12:17:14.730 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.731 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.731 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.731 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.731 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.731 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.731 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.731 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.732 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.732 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-04-25T12:17:14.732 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-04-25T12:17:14.732 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.732 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.732 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.732 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.732 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.733 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.733 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T12:17:14.733 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T12:17:14.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.733 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.733 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.734 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.734 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.734 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.734 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.734 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.734 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.734 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.735 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.735 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.735 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.735 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.735 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.735 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.735 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.735 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.736 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.736 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.736 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.736 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.736 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.737 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.737 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.737 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.737 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.737 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.737 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.738 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.738 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.738 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.738 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.738 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.739 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.739 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.739 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.739 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.739 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.739 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.739 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.740 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.740 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.740 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.740 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.740 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.740 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.740 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.741 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.741 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.741 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.741 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.741 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.741 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.741 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.741 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.741 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.742 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.742 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.742 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.742 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-04-25T12:17:14.742 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-04-25T12:17:14.742 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.742 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.743 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.743 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.743 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.743 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.743 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.743 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.743 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.743 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.744 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.744 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T12:17:14.744 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T12:17:14.744 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.744 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.744 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.745 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-04-25T12:17:14.745 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.745 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.745 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.745 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.745 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.745 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.745 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.746 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-04-25T12:17:14.746 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-04-25T12:17:14.746 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:14.746 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.746 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.746 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-04-25T12:17:14.746 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.747 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.747 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.747 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.747 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.747 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.747 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.747 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T12:17:14.748 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T12:17:14.748 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.748 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.748 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.748 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T12:17:14.748 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.748 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.748 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.749 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T12:17:14.749 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.749 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.749 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.749 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T12:17:14.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T12:17:14.749 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.750 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.750 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.750 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-04-25T12:17:14.750 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.750 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.750 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.750 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T12:17:14.750 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.751 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.751 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.751 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-04-25T12:17:14.751 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-04-25T12:17:14.751 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.751 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.751 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.751 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-04-25T12:17:14.752 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.752 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.752 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.752 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.752 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.752 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.752 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.753 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T12:17:14.753 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T12:17:14.753 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.753 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.753 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.753 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-04-25T12:17:14.753 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.753 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.753 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.754 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-04-25T12:17:14.754 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-04-25T12:17:14.754 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.754 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.754 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.754 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.754 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.755 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T12:17:14.755 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T12:17:14.755 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.755 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.755 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.755 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T12:17:14.755 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T12:17:14.755 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T12:17:14.755 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.756 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.756 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.756 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.756 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.756 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.756 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.756 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.757 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-25T12:17:14.757 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.757 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.757 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.757 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.757 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.757 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.757 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.757 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.758 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.758 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.758 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.758 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.758 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.758 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.758 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.759 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.759 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.759 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.759 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.759 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.759 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.759 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.760 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.760 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.760 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.760 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.760 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.760 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.760 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.760 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.760 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.761 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.761 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.761 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.761 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.761 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.761 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.762 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.762 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.762 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.762 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.762 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.762 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.762 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.762 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.762 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.763 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.763 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.763 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.763 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.763 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.763 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.763 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.764 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.764 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.764 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.764 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.764 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.764 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.765 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.765 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.765 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.765 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.765 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.765 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.765 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-04-25T12:17:14.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-04-25T12:17:14.766 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.766 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.766 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.766 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:14.766 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.766 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.767 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.767 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.767 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.767 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-04-25T12:17:14.767 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T12:17:14.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.768 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.768 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.768 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.768 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.768 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.768 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.768 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.768 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.769 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-04-25T12:17:14.769 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-04-25T12:17:14.769 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.769 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.770 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.770 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.770 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.770 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.770 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.770 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.770 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.770 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.771 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.771 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.771 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.771 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-04-25T12:17:14.771 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.771 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.772 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.772 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-04-25T12:17:14.772 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-04-25T12:17:14.772 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.772 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.772 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.772 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.772 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.773 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.773 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.773 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.773 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.773 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.773 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T12:17:14.773 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T12:17:14.774 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.774 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.774 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.774 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.774 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.774 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.774 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.774 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.775 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.775 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.775 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.775 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.775 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.775 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.775 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.776 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.776 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.776 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.776 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.776 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.776 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.776 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.776 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.777 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.777 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.777 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.777 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.777 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.777 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.777 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.777 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.778 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.778 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.778 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.778 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.778 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.778 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.778 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.778 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.779 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.779 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.779 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.779 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.779 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.780 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.780 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.780 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.780 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.780 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.780 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.780 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.780 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.781 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.781 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.781 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.781 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.781 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.781 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.781 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.782 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.782 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.782 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.782 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.782 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.783 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.783 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.783 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.783 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.783 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.783 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.784 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T12:17:14.784 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T12:17:14.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.784 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.784 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.784 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.785 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.785 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.785 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.785 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.785 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T12:17:14.785 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T12:17:14.785 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.786 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.786 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.786 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.786 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.786 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.786 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.786 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.786 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.787 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.787 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.787 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.787 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.787 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.787 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-04-25T12:17:14.787 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.788 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.788 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.788 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.788 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.788 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.788 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.788 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.788 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.789 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.789 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.789 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.789 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.789 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.789 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.789 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.789 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.789 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.790 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.790 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.790 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.790 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.790 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.790 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.790 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.791 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.791 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.791 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.791 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.791 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.791 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.791 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.791 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.792 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.792 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.792 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.792 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.792 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.792 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.793 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.793 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.793 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.793 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.793 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.793 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.793 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.794 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.794 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.794 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.794 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.794 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.794 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.794 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.795 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.795 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.795 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.795 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.795 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.795 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.795 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.796 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.796 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.796 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.796 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.796 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.796 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.796 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.796 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-04-25T12:17:14.797 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.797 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.797 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.797 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.797 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.797 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.798 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.798 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.798 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.798 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.798 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.798 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.798 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.798 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.799 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.799 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.799 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.799 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.799 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.799 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.799 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.799 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.800 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.800 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.800 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.800 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.800 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.800 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.800 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.801 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.801 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.801 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.801 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.801 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.801 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.801 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.802 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.802 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.802 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.802 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.802 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.802 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.802 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.803 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.803 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.803 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.803 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.803 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.803 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.803 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.803 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.804 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.804 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.804 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.804 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.804 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.804 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.804 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.804 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.805 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.805 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.805 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.805 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.805 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.805 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.805 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.806 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-04-25T12:17:14.806 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-04-25T12:17:14.806 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.806 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.806 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-04-25T12:17:14.806 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.806 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.807 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T12:17:14.807 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.807 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.807 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.807 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.807 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.807 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.807 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-04-25T12:17:14.808 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.808 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.808 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.808 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.808 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.808 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.808 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.808 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.809 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.809 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.809 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.809 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.809 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.809 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.809 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.809 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.809 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.810 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.810 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.810 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.810 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.810 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.810 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.810 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.810 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.811 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.811 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.811 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.811 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.811 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.811 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.811 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.811 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.811 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.812 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.812 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.812 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.812 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.812 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.812 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.812 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.812 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.813 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.813 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.813 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.813 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.813 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.813 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.813 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.813 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.813 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.814 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.814 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.814 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.814 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.814 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.814 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.814 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.815 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.815 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.815 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.815 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.815 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.815 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.815 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.815 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.816 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.816 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.816 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-04-25T12:17:14.816 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-04-25T12:17:14.816 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.816 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.816 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.817 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.817 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.817 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.817 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.817 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.817 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.817 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.818 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-04-25T12:17:14.818 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-04-25T12:17:14.818 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.818 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.818 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.818 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-04-25T12:17:14.818 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.818 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.819 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.819 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.819 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.819 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.819 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.819 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-04-25T12:17:14.819 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-04-25T12:17:14.820 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.820 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.820 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.820 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.820 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.820 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.820 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.820 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.820 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.821 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.821 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.821 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-04-25T12:17:14.821 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-04-25T12:17:14.821 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.821 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.821 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.822 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.822 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.822 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.822 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.822 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.822 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.822 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.822 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.823 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-04-25T12:17:14.823 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-04-25T12:17:14.823 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:14.823 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.823 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.823 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.823 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.824 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.824 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.824 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.824 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.824 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.824 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-04-25T12:17:14.824 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-04-25T12:17:14.825 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.825 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.825 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.825 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.825 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.825 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.825 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.825 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.825 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.826 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.826 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.826 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-04-25T12:17:14.826 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-04-25T12:17:14.826 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.826 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.826 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.827 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.827 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.827 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.827 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.827 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.827 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.827 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.827 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.828 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.828 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-04-25T12:17:14.828 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-04-25T12:17:14.828 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.828 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.828 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.828 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.828 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.828 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.829 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.829 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.829 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.829 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.829 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.829 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-04-25T12:17:14.829 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-04-25T12:17:14.830 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.830 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.830 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.830 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.830 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T12:17:14.830 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-04-25T12:17:14.830 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.830 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.831 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.831 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.831 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.831 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-04-25T12:17:14.831 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-04-25T12:17:14.831 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.831 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.831 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.832 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.832 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.832 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.832 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.832 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.832 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.832 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.832 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.833 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-04-25T12:17:14.833 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-04-25T12:17:14.833 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.833 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.833 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.833 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.833 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.833 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.833 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.834 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.834 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.834 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.834 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.834 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-04-25T12:17:14.834 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-04-25T12:17:14.834 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.835 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.835 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.835 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.835 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.835 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.835 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.835 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.835 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.836 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.836 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.836 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.836 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.836 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.836 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T12:17:14.836 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.836 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.837 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.837 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-04-25T12:17:14.837 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-04-25T12:17:14.837 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.837 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.837 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.837 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.837 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.838 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.838 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.838 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T12:17:14.838 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.838 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.838 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.838 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-04-25T12:17:14.839 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-04-25T12:17:14.839 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.839 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.839 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.839 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.839 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.839 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.840 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.840 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-04-25T12:17:14.840 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.840 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.840 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.840 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.840 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.841 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.841 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.841 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.841 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.841 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.841 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.841 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.842 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.842 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.842 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.842 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.842 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.842 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.842 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.843 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.843 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.843 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.843 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.843 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.843 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.843 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.844 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.844 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.844 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.844 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.844 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.844 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.844 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.845 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.845 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.845 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.845 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.845 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.845 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.845 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.846 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.846 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.846 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.846 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.846 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.846 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.846 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.847 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.847 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.847 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.847 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.847 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.847 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.847 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.847 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.848 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.848 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.849 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.849 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.850 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.850 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.850 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.850 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.850 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.850 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.850 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.851 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.851 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.851 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.851 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.851 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.851 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.851 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-04-25T12:17:14.851 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.852 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.852 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.852 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.852 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.852 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.852 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.852 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.852 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.853 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.853 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.853 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.853 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.853 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.853 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.853 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.853 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.854 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.854 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.854 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.854 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.854 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.854 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.854 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.854 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.855 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.855 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.855 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.855 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.855 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.855 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.855 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.856 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.856 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.856 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.856 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.856 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.856 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.856 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.857 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.857 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.857 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.857 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.857 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.857 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.857 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.857 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.857 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.858 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.858 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.858 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.858 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.858 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.858 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.858 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.858 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.859 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.859 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.859 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.859 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.859 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.859 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.859 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.859 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.859 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.860 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.860 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.860 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.860 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.860 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.860 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.860 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.860 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.860 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.861 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-04-25T12:17:14.861 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.861 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.861 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.861 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.862 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.862 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.862 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.862 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.862 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.862 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.862 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.862 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.863 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.863 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.863 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.863 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.863 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.863 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.863 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.863 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.864 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.864 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.864 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.864 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.864 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.864 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.864 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.865 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.865 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.865 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.865 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.865 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.865 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.865 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.865 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.866 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.866 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.866 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.866 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.866 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.866 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.866 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.866 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.867 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.867 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.867 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.867 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.867 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.867 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.867 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.867 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.868 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.868 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.868 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.868 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.868 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.868 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.868 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-04-25T12:17:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-04-25T12:17:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-04-25T12:17:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T12:17:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-04-25T12:17:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.884 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.884 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.884 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.884 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-04-25T12:17:14.884 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-04-25T12:17:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-04-25T12:17:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.886 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.886 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.886 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.886 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T12:17:14.886 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.886 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.886 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-04-25T12:17:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-04-25T12:17:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T12:17:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-04-25T12:17:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-04-25T12:17:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.889 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.889 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.889 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.889 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.889 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.889 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T12:17:14.889 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.890 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.890 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.890 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-04-25T12:17:14.890 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-04-25T12:17:14.891 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.891 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.891 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.891 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.891 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.891 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.891 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.891 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T12:17:14.892 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.892 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.892 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.892 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-04-25T12:17:14.892 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-04-25T12:17:14.892 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T12:17:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.894 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.894 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-04-25T12:17:14.894 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-04-25T12:17:14.894 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.894 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.894 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.894 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.895 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.895 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.895 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.895 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.895 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.895 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.895 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.895 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-04-25T12:17:14.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-04-25T12:17:14.896 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.896 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.896 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.896 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.896 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.896 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.896 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.896 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.897 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.897 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.897 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.897 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-04-25T12:17:14.897 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-04-25T12:17:14.897 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.897 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.897 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.898 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T12:17:14.898 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.898 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.898 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.898 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.898 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.898 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.899 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.899 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.899 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T12:17:14.899 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T12:17:14.899 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.899 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.899 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.899 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.900 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.900 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.900 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.900 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.900 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.900 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.900 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:14.901 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:14.901 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.901 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.901 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-04-25T12:17:14.901 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-04-25T12:17:14.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.901 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.901 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.902 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.902 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.902 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.902 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-04-25T12:17:14.902 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-04-25T12:17:14.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.903 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.903 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-04-25T12:17:14.903 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.903 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.903 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.903 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.903 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.904 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.904 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-04-25T12:17:14.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-04-25T12:17:14.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.904 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.904 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.905 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.905 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.905 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.905 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.905 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.905 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.906 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-04-25T12:17:14.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-04-25T12:17:14.906 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.906 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.906 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.906 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.906 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.906 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.907 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.907 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.907 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.907 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.907 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.907 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.907 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.908 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.908 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.908 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.908 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.908 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.908 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.908 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.908 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.909 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.909 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.909 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.909 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.909 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.909 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.909 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.909 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.910 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.910 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.910 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.910 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.910 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.910 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.911 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.911 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.911 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.911 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.912 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.912 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.912 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.912 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.912 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.912 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.912 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.912 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.912 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.913 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.913 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.913 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.913 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.913 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.913 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.913 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.913 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.914 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.914 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.914 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.914 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.914 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.915 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.915 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.915 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.915 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.915 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.915 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.916 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-04-25T12:17:14.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-04-25T12:17:14.916 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.916 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.916 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.916 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.916 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.916 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.917 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.917 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.917 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.917 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.917 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.917 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-04-25T12:17:14.917 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-04-25T12:17:14.918 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.918 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.918 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.918 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.918 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.919 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.919 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.919 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.919 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.919 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.919 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-04-25T12:17:14.919 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-04-25T12:17:14.919 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.920 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.920 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.920 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T12:17:14.920 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.920 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.920 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.920 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.921 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.921 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.921 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.921 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.921 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-04-25T12:17:14.921 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.922 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.922 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.922 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.922 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.922 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.922 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.922 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.922 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.923 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.923 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.923 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.923 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.923 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.923 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.923 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.923 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.924 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.924 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.924 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.924 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.924 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.924 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.924 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.924 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.924 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.925 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.925 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.925 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.925 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.925 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.926 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.926 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.926 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.926 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.926 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.927 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.927 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.927 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.927 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.928 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.928 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.928 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.928 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.928 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.928 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.928 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.928 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.928 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.929 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.929 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.929 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.929 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.929 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.929 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.929 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.930 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.930 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.930 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.930 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.930 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.930 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.930 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.931 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.931 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.931 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.931 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.931 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.931 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.931 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.932 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.932 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-25T12:17:14.932 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.932 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.932 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.932 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.933 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.933 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.933 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.933 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.933 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.933 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.934 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.934 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.934 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.934 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.935 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.935 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.935 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.935 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.935 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.935 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.936 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.936 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.936 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.936 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.936 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.936 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.936 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.936 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.936 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.937 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.937 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.937 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.937 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.937 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.937 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.937 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.938 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.938 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.938 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.938 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.938 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.938 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.938 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.938 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.939 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.939 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.939 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.939 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.939 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.939 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.939 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.939 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.940 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.940 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.940 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.940 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.940 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.940 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.941 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.941 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.941 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.941 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.941 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-04-25T12:17:14.942 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-04-25T12:17:14.942 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.942 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.942 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.942 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-04-25T12:17:14.942 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.942 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.942 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.943 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T12:17:14.943 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.943 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.943 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.943 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-04-25T12:17:14.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-04-25T12:17:14.944 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.944 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.944 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.944 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.944 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.944 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.945 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.945 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-04-25T12:17:14.945 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.945 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.945 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.945 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-04-25T12:17:14.945 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-04-25T12:17:14.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.946 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.946 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.946 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:14.946 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.946 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.946 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.946 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T12:17:14.946 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.947 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.947 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.947 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-04-25T12:17:14.947 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T12:17:14.947 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.947 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.948 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.948 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.948 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.948 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T12:17:14.948 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.948 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.948 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.948 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.948 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-04-25T12:17:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.949 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-04-25T12:17:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-04-25T12:17:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-04-25T12:17:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.950 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.951 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-04-25T12:17:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-04-25T12:17:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.952 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:14.953 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.954 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.956 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.957 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.958 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.959 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.960 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.960 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.960 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.960 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.960 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.960 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.960 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.961 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.961 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.961 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.961 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.961 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.961 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.961 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.961 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.962 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.962 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.962 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.962 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.962 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.962 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.962 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.963 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.963 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.963 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.963 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.963 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.964 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.964 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-04-25T12:17:14.964 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-04-25T12:17:14.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.964 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.965 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.965 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.965 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.965 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.965 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.965 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.966 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-04-25T12:17:14.966 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-04-25T12:17:14.966 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.966 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.966 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.966 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-04-25T12:17:14.966 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.967 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.967 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.967 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.967 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.967 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.967 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-04-25T12:17:14.968 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-04-25T12:17:14.968 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.968 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:14.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T12:17:14.968 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.968 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.968 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.969 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.969 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.969 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.969 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.970 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T12:17:14.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.970 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 25033, 2024-04-25T12:17:14.970 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "z", 2024-04-25T12:17:14.970 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-04-25T12:17:14.970 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-04-25T12:17:14.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.971 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-04-25T12:17:14.971 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.971 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.971 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.971 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:14.971 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:14.971 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:14.971 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.972 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.972 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:14.972 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.972 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.972 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.972 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T12:17:14.973 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.973 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.973 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.973 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:14.973 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:14.973 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.973 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.974 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.974 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.974 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.974 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.974 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.974 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.974 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.975 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.975 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.975 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.975 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.975 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.975 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.975 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.975 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.976 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:14.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:14.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.976 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.976 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.976 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.976 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.977 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.977 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.977 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.977 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.978 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:14.978 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:14.978 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.978 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.978 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:14.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.979 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:14.979 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:14.979 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:14.979 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:14.979 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:14.979 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:14.979 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:14.979 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:14.980 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.980 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.980 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.980 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:14.980 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:14.980 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.981 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:14.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.981 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.981 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.981 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:14.981 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.982 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.982 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.982 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-04-25T12:17:14.982 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-04-25T12:17:14.982 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.982 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.982 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.983 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.983 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.983 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.983 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.983 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-04-25T12:17:14.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.984 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-04-25T12:17:14.984 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-04-25T12:17:14.984 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.984 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.984 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.984 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-04-25T12:17:14.984 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.985 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.985 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.985 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-04-25T12:17:14.985 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.985 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.985 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.986 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-04-25T12:17:14.986 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-04-25T12:17:14.986 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.986 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.986 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.986 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.986 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.987 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.987 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.987 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-04-25T12:17:14.987 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.987 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.987 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.987 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-04-25T12:17:14.987 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-04-25T12:17:14.988 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.988 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.988 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.988 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.988 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.988 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.988 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.988 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-04-25T12:17:14.989 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.989 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.989 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.989 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-04-25T12:17:14.989 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-04-25T12:17:14.989 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:14.989 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.989 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.990 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-04-25T12:17:14.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.990 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.990 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.990 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-04-25T12:17:14.990 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.991 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.991 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.991 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-04-25T12:17:14.991 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-04-25T12:17:14.991 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.991 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.992 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.992 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.992 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.992 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.992 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-04-25T12:17:14.992 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.993 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.993 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.993 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-04-25T12:17:14.993 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-04-25T12:17:14.993 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.993 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.994 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.994 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.994 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.994 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.994 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.994 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T12:17:14.994 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.994 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.995 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.995 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-04-25T12:17:14.995 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-04-25T12:17:14.995 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.995 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.995 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:14.995 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.996 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.996 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.996 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-04-25T12:17:14.996 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.996 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.996 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.996 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:14.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:14.997 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-04-25T12:17:14.997 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:14.997 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:14.997 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:14.997 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-04-25T12:17:14.997 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-04-25T12:17:14.997 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:14.998 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.998 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:14.998 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:14.998 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:14.998 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:14.998 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:14.998 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T12:17:14.998 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:14.999 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:14.999 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:14.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:14.999 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-04-25T12:17:14.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-04-25T12:17:14.999 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:14.999 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:14.999 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.000 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-04-25T12:17:15.000 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.000 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.000 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.000 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T12:17:15.000 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T12:17:15.000 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.001 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.001 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-04-25T12:17:15.001 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-04-25T12:17:15.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:15.001 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.001 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:15.002 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T12:17:15.002 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T12:17:15.002 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.002 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T12:17:15.002 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T12:17:15.002 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.003 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-04-25T12:17:15.003 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-04-25T12:17:15.003 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:15.003 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.003 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.003 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-04-25T12:17:15.003 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T12:17:15.003 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-04-25T12:17:15.004 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.004 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T12:17:15.004 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.004 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.004 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.004 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-04-25T12:17:15.004 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-04-25T12:17:15.004 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.005 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.005 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T12:17:15.005 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.005 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.005 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.005 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T12:17:15.006 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T12:17:15.006 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.006 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.006 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-04-25T12:17:15.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-04-25T12:17:15.006 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:15.007 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.007 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.007 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-04-25T12:17:15.007 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-04-25T12:17:15.007 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-04-25T12:17:15.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.007 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-04-25T12:17:15.007 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T12:17:15.008 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.008 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.008 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-04-25T12:17:15.008 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-04-25T12:17:15.008 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.008 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.008 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.008 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-04-25T12:17:15.009 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.009 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.009 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.009 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T12:17:15.009 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T12:17:15.009 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.009 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.010 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-04-25T12:17:15.010 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-04-25T12:17:15.010 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:15.010 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.010 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.010 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:15.010 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T12:17:15.010 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T12:17:15.011 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.011 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T12:17:15.011 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T12:17:15.011 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.011 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.011 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.011 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.012 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.012 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.012 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.012 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.012 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.012 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.012 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.013 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.013 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.013 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.013 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.013 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.013 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.013 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.013 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.014 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.014 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.014 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.014 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.014 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.014 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.014 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.015 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.015 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.015 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.015 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.015 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.015 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.015 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.016 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.016 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.016 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.016 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.016 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.017 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.017 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.017 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.017 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.017 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.017 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.017 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.017 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.017 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.018 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.018 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.018 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.018 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.018 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.019 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.020 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.020 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.020 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-04-25T12:17:15.020 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-04-25T12:17:15.020 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:15.020 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.020 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.020 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:15.021 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.021 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.021 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.021 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T12:17:15.021 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.021 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.021 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.022 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T12:17:15.022 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T12:17:15.022 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.022 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.022 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.022 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-04-25T12:17:15.022 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.022 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.023 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.023 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-04-25T12:17:15.023 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-04-25T12:17:15.023 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-04-25T12:17:15.023 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-04-25T12:17:15.023 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-04-25T12:17:15.023 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.023 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-04-25T12:17:15.024 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.024 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.024 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-04-25T12:17:15.024 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-04-25T12:17:15.024 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.024 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.024 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.025 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.025 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.025 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.025 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T12:17:15.025 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.025 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.025 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.026 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:15.026 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:15.026 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.026 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.026 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.026 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:15.026 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.027 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.027 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.027 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T12:17:15.027 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.027 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.027 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.027 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-04-25T12:17:15.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-04-25T12:17:15.028 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.028 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.028 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.028 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T12:17:15.028 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T12:17:15.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.028 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.028 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T12:17:15.029 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T12:17:15.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.029 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.029 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-04-25T12:17:15.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-04-25T12:17:15.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:15.029 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.030 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:15.030 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.030 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.030 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.030 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T12:17:15.030 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.031 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.031 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.031 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.031 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.031 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-04-25T12:17:15.031 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.032 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.032 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.032 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-04-25T12:17:15.032 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-04-25T12:17:15.032 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:15.032 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.032 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.032 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T12:17:15.033 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.033 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.033 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.033 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T12:17:15.033 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.033 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.033 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.033 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-04-25T12:17:15.034 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-04-25T12:17:15.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.034 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.034 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-04-25T12:17:15.034 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.034 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T12:17:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-04-25T12:17:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-04-25T12:17:15.035 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.036 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.036 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-04-25T12:17:15.036 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.036 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.036 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.036 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T12:17:15.036 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.037 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.037 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-04-25T12:17:15.037 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-04-25T12:17:15.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.037 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.037 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.038 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.038 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.038 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.038 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T12:17:15.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T12:17:15.038 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.038 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.039 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-04-25T12:17:15.039 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-04-25T12:17:15.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.039 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.039 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:15.039 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.039 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T12:17:15.040 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.040 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.040 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.040 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-04-25T12:17:15.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T12:17:15.040 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:15.041 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.041 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-04-25T12:17:15.041 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.041 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.041 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.041 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T12:17:15.041 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.041 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.042 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.042 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-04-25T12:17:15.042 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-04-25T12:17:15.042 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.042 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.042 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.043 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.043 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.043 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.043 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-04-25T12:17:15.043 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-04-25T12:17:15.043 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-04-25T12:17:15.043 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.043 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T12:17:15.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.044 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.044 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.044 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-04-25T12:17:15.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-04-25T12:17:15.044 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.045 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.045 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.045 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.045 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.045 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.045 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.045 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T12:17:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-04-25T12:17:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-04-25T12:17:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.046 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.047 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.047 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.047 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.047 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.047 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.047 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T12:17:15.047 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.047 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.048 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.048 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-04-25T12:17:15.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-04-25T12:17:15.048 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.048 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.049 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.049 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.049 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.049 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T12:17:15.049 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.049 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.049 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.049 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-04-25T12:17:15.050 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-04-25T12:17:15.050 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.050 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.050 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.050 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T12:17:15.050 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.050 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.053 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.054 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T12:17:15.054 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.054 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.054 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.054 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-04-25T12:17:15.054 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-04-25T12:17:15.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.055 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.055 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T12:17:15.055 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.055 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.055 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.055 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-04-25T12:17:15.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.056 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.056 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.056 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-04-25T12:17:15.056 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-04-25T12:17:15.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.056 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.057 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T12:17:15.057 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.057 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.057 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.057 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-04-25T12:17:15.057 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.057 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.058 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.058 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-04-25T12:17:15.058 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-04-25T12:17:15.058 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.058 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.058 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T12:17:15.059 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.059 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.059 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.059 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T12:17:15.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.059 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.059 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.060 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-04-25T12:17:15.060 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-04-25T12:17:15.060 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.060 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.060 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T12:17:15.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.061 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.061 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-04-25T12:17:15.061 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.061 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.062 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-04-25T12:17:15.062 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-04-25T12:17:15.062 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.062 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T12:17:15.062 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.062 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.063 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.063 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-04-25T12:17:15.063 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.063 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.063 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-04-25T12:17:15.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-04-25T12:17:15.063 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.064 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T12:17:15.064 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.064 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.064 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-04-25T12:17:15.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.065 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.065 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-04-25T12:17:15.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-04-25T12:17:15.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.066 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T12:17:15.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.066 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.066 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-04-25T12:17:15.066 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.067 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.067 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-04-25T12:17:15.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-04-25T12:17:15.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.067 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T12:17:15.068 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.068 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.068 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.068 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-04-25T12:17:15.068 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.068 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.068 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.069 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-04-25T12:17:15.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-04-25T12:17:15.069 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.069 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.069 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.069 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T12:17:15.069 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.070 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-04-25T12:17:15.070 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.070 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.070 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.071 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-04-25T12:17:15.071 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-04-25T12:17:15.071 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.071 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.071 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.071 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T12:17:15.071 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.071 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.072 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T12:17:15.072 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.072 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.072 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.072 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.072 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-04-25T12:17:15.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-04-25T12:17:15.072 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.073 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.073 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T12:17:15.073 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.073 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.073 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-04-25T12:17:15.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.074 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-04-25T12:17:15.074 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-04-25T12:17:15.074 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.074 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.075 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.075 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T12:17:15.075 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.075 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.075 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.075 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T12:17:15.075 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.075 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.076 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.076 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-04-25T12:17:15.076 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-04-25T12:17:15.076 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.076 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.076 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.076 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T12:17:15.076 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.077 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.077 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-04-25T12:17:15.077 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.077 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.077 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.077 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-04-25T12:17:15.078 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-04-25T12:17:15.078 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.078 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.078 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.078 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T12:17:15.078 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.079 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.079 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.079 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-04-25T12:17:15.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.079 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.079 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.079 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-04-25T12:17:15.079 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-04-25T12:17:15.080 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.080 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.080 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T12:17:15.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.080 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.080 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-04-25T12:17:15.081 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.081 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.081 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-04-25T12:17:15.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-04-25T12:17:15.081 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.082 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.082 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.082 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T12:17:15.082 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-04-25T12:17:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-04-25T12:17:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.084 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:15.084 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.084 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.084 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.084 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T12:17:15.084 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.084 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.084 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.085 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-04-25T12:17:15.085 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T12:17:15.085 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.085 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.085 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-04-25T12:17:15.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.086 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.086 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.086 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T12:17:15.086 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.086 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.086 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.087 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.087 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-04-25T12:17:15.087 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-04-25T12:17:15.087 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.087 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.087 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.087 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-04-25T12:17:15.087 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.088 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.088 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.088 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T12:17:15.088 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.088 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.088 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-04-25T12:17:15.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-04-25T12:17:15.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-04-25T12:17:15.089 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.089 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-04-25T12:17:15.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.090 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.090 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-04-25T12:17:15.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-04-25T12:17:15.091 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.091 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.091 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.091 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.091 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.091 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.092 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T12:17:15.092 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.092 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.092 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.092 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-04-25T12:17:15.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-04-25T12:17:15.092 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.093 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.093 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.093 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:15.093 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.093 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.093 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-04-25T12:17:15.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.094 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.094 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.094 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.094 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-04-25T12:17:15.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-04-25T12:17:15.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.094 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.095 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.095 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T12:17:15.095 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.095 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.095 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.095 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T12:17:15.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.095 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.096 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-04-25T12:17:15.096 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-04-25T12:17:15.096 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.096 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.096 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.096 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:15.096 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.097 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-04-25T12:17:15.097 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.097 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.097 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.097 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-04-25T12:17:15.097 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-04-25T12:17:15.098 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.098 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.098 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.098 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.098 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.098 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T12:17:15.099 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.099 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.099 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.099 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-04-25T12:17:15.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-04-25T12:17:15.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.100 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.100 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.100 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.100 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.100 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T12:17:15.100 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.101 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.101 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.101 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.101 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.101 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.101 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.101 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.101 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.102 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.102 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.102 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.102 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.102 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.102 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.102 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.103 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.103 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.103 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.103 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.103 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-04-25T12:17:15.104 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-04-25T12:17:15.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.104 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.104 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.104 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.104 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.104 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.104 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T12:17:15.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.105 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.105 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.105 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.105 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.105 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.105 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.106 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.106 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.106 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.106 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.106 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T12:17:15.106 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.106 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.107 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.107 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.107 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.107 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.107 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.107 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.108 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.108 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.108 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.108 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.108 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.108 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.108 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.109 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.109 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.109 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.109 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.110 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.110 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.110 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.110 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.110 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.110 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.110 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.111 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.111 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.111 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.111 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T12:17:15.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T12:17:15.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.112 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.112 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T12:17:15.112 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.112 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.112 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.113 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T12:17:15.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T12:17:15.113 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.113 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.113 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.113 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-04-25T12:17:15.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.113 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.114 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.114 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T12:17:15.114 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.114 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.114 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-04-25T12:17:15.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-04-25T12:17:15.114 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.115 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.115 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:15.115 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.115 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.115 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-04-25T12:17:15.115 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-04-25T12:17:15.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-04-25T12:17:15.116 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.116 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.116 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.116 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:15.117 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.117 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.117 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T12:17:15.117 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.117 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.117 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.117 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-04-25T12:17:15.118 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-04-25T12:17:15.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.118 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.118 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.118 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.118 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.118 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.118 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-04-25T12:17:15.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.119 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T12:17:15.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T12:17:15.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.120 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.120 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.120 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-04-25T12:17:15.120 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.120 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.120 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.121 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-04-25T12:17:15.121 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-04-25T12:17:15.121 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.121 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T12:17:15.121 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.121 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.121 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-04-25T12:17:15.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-04-25T12:17:15.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.122 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.122 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-04-25T12:17:15.122 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.122 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T12:17:15.123 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.123 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.123 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.123 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-04-25T12:17:15.123 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-04-25T12:17:15.123 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.123 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.124 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T12:17:15.124 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.124 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.124 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.124 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T12:17:15.124 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.125 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.125 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-04-25T12:17:15.125 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-04-25T12:17:15.125 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.125 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.125 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.125 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.125 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.126 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.126 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T12:17:15.126 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.126 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-04-25T12:17:15.127 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-04-25T12:17:15.127 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.127 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.127 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.127 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.127 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.127 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.127 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.128 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T12:17:15.128 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.128 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.128 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.128 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-04-25T12:17:15.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-04-25T12:17:15.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.129 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.129 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.129 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.129 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T12:17:15.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.130 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.130 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.130 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-04-25T12:17:15.130 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-04-25T12:17:15.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.131 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.131 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.131 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.131 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T12:17:15.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.132 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.132 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-04-25T12:17:15.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-04-25T12:17:15.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.133 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.133 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.133 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T12:17:15.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.133 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.133 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.133 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-04-25T12:17:15.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-04-25T12:17:15.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.134 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-04-25T12:17:15.134 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.134 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.135 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.135 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-04-25T12:17:15.135 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.136 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.136 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.136 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-04-25T12:17:15.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-04-25T12:17:15.137 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.137 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.137 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.138 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.138 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.139 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T12:17:15.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.140 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-04-25T12:17:15.140 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-04-25T12:17:15.141 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.141 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.142 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.142 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.142 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.143 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T12:17:15.143 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.143 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.143 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.144 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-04-25T12:17:15.144 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-04-25T12:17:15.145 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.145 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.146 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.146 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.146 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.147 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T12:17:15.147 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.147 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.147 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.148 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-04-25T12:17:15.148 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-04-25T12:17:15.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.149 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.149 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.150 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.150 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T12:17:15.151 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.151 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.151 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.152 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-04-25T12:17:15.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-04-25T12:17:15.152 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.153 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.154 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.154 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.154 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.154 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T12:17:15.155 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.155 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.155 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.155 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-04-25T12:17:15.155 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-04-25T12:17:15.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.156 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.156 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.156 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.156 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T12:17:15.157 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.157 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.157 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.157 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.157 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.157 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-04-25T12:17:15.158 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.158 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.158 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.158 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.158 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.158 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.158 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.159 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.159 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.159 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.159 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.159 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.159 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.160 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.160 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.160 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.160 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.160 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.160 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.161 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.161 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.161 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.161 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.161 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.161 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.162 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.162 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.162 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.162 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.162 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.162 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.162 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.163 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.163 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.163 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.163 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.163 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.163 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.163 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.163 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.163 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.164 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.164 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.164 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.164 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.164 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.164 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.164 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.164 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.165 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.165 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.165 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.165 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.166 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.166 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.166 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.166 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.166 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.166 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-04-25T12:17:15.166 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-04-25T12:17:15.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.167 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.167 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-04-25T12:17:15.167 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.167 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.167 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.167 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T12:17:15.168 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.168 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.168 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.168 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-04-25T12:17:15.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-04-25T12:17:15.168 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.168 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.169 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.169 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-04-25T12:17:15.169 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.169 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.169 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.169 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T12:17:15.169 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.169 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.170 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.170 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.170 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.170 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-04-25T12:17:15.170 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.171 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.171 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.171 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T12:17:15.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T12:17:15.171 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.171 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:15.172 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.172 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.172 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.172 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.172 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.172 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.172 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.172 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T12:17:15.173 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T12:17:15.173 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.173 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.173 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.173 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.174 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.174 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.174 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.174 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.174 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.174 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T12:17:15.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T12:17:15.175 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.175 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.175 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.175 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.175 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.175 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.176 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.176 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.176 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-04-25T12:17:15.176 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T12:17:15.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.177 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.177 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.177 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.177 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.177 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.177 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.178 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.178 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.178 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T12:17:15.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T12:17:15.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.179 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.179 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.179 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.179 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.179 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.180 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.180 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.180 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.180 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T12:17:15.180 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T12:17:15.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.181 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.182 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.182 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T12:17:15.182 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T12:17:15.182 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.183 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.183 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.184 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T12:17:15.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T12:17:15.184 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.184 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.185 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.185 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.185 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.185 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.185 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T12:17:15.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T12:17:15.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.186 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.187 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.187 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T12:17:15.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T12:17:15.188 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.188 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.188 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.188 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.188 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.188 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.189 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T12:17:15.189 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T12:17:15.189 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.189 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.190 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.190 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.191 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T12:17:15.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T12:17:15.191 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.191 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.191 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.192 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.192 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.192 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.192 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.193 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.193 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T12:17:15.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T12:17:15.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.193 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.193 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.193 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.195 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T12:17:15.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T12:17:15.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.195 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.195 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.195 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.196 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.196 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.196 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.196 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.196 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T12:17:15.196 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T12:17:15.196 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.196 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.197 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T12:17:15.197 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.197 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.197 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.198 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T12:17:15.198 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T12:17:15.198 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.198 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.198 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.199 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.199 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.199 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.199 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.199 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-04-25T12:17:15.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-04-25T12:17:15.200 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.200 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.200 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.200 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.200 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.200 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.200 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.201 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.201 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.201 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.201 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T12:17:15.201 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T12:17:15.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.202 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.202 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T12:17:15.202 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.202 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.203 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.203 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T12:17:15.203 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T12:17:15.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.204 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.204 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.204 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.204 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T12:17:15.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T12:17:15.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.205 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.205 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.206 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.206 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.206 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T12:17:15.206 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T12:17:15.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.206 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.207 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.207 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.207 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.207 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.208 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T12:17:15.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T12:17:15.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.208 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.208 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.208 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.208 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.209 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.210 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T12:17:15.210 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T12:17:15.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.210 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.210 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.210 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-04-25T12:17:15.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.211 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.211 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.211 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.211 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.211 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T12:17:15.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T12:17:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-04-25T12:17:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T12:17:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T12:17:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.214 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.214 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.214 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.214 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.214 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.214 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.215 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T12:17:15.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T12:17:15.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.215 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.215 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.215 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.215 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.216 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.216 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.216 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.216 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T12:17:15.216 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T12:17:15.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.217 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.217 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.217 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.217 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.218 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.218 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.218 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T12:17:15.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T12:17:15.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.218 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.218 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.219 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.220 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T12:17:15.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T12:17:15.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.220 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.221 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.221 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.221 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.222 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T12:17:15.222 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T12:17:15.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.222 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.222 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.222 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.222 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.223 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.223 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T12:17:15.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T12:17:15.224 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.224 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.224 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.225 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.225 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.225 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T12:17:15.225 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T12:17:15.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.225 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.226 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.226 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.226 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.226 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.226 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.226 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.227 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T12:17:15.227 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T12:17:15.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.227 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.227 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.228 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.228 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.228 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.228 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T12:17:15.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T12:17:15.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.229 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.229 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.229 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.229 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.230 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.230 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.230 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.230 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T12:17:15.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T12:17:15.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.231 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.231 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T12:17:15.231 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.231 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.231 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.232 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.232 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T12:17:15.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T12:17:15.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:15.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.233 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.233 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.233 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.233 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T12:17:15.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T12:17:15.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.234 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.234 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-04-25T12:17:15.234 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.234 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.234 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.235 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.235 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T12:17:15.235 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T12:17:15.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.235 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.236 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.236 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-04-25T12:17:15.236 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.236 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.236 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.237 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T12:17:15.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T12:17:15.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.237 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.237 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.237 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.238 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.238 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.238 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.238 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.238 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T12:17:15.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T12:17:15.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.239 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.239 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T12:17:15.239 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.239 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.239 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.240 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.240 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.240 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.240 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T12:17:15.240 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T12:17:15.240 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.240 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.241 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.241 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.241 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.241 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.241 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.241 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.241 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.241 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.242 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T12:17:15.242 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T12:17:15.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.242 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.242 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.242 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.242 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.243 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.243 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.243 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.243 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T12:17:15.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T12:17:15.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.244 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.245 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.245 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.245 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T12:17:15.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T12:17:15.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.246 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.246 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.246 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:15.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.246 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.246 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.246 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.247 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.247 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.247 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T12:17:15.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T12:17:15.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.247 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.247 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:15.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.248 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.249 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T12:17:15.249 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T12:17:15.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T12:17:15.249 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.250 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.250 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.250 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.251 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-04-25T12:17:15.251 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-04-25T12:17:15.251 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.251 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.251 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.251 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.252 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.252 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-04-25T12:17:15.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-04-25T12:17:15.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.254 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.258 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.258 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-04-25T12:17:15.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-04-25T12:17:15.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.259 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.260 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-04-25T12:17:15.260 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.260 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-04-25T12:17:15.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-04-25T12:17:15.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.261 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-04-25T12:17:15.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.261 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.261 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.262 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.262 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T12:17:15.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T12:17:15.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.262 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.262 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.263 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.263 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.263 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.263 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.263 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.263 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.263 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.264 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.264 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.264 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.264 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.264 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.264 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.265 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.265 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.265 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.265 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.265 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.265 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.266 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.266 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.266 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.266 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.267 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.267 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.267 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.267 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.267 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.268 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.268 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.268 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.268 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.268 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.268 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.269 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.269 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.269 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.269 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.269 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.269 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.269 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.269 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.270 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.270 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.270 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.270 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.271 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.271 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-04-25T12:17:15.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-04-25T12:17:15.272 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.272 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.272 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.272 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.272 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.272 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.273 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-04-25T12:17:15.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.273 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.273 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.273 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-04-25T12:17:15.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T12:17:15.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.274 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.274 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.275 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.275 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.275 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T12:17:15.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T12:17:15.275 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.275 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.276 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T12:17:15.276 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.276 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T12:17:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T12:17:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-04-25T12:17:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T12:17:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T12:17:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-04-25T12:17:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-04-25T12:17:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-04-25T12:17:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.282 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T12:17:15.282 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T12:17:15.282 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.282 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.282 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.282 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-04-25T12:17:15.282 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.283 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.283 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-04-25T12:17:15.283 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.283 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.283 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.283 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.284 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.284 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T12:17:15.284 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T12:17:15.284 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T12:17:15.285 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T12:17:15.285 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T12:17:15.285 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.285 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.286 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-04-25T12:17:15.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-04-25T12:17:15.286 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.286 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.287 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.287 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.287 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.287 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.287 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.287 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.288 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.288 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.288 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-25T12:17:15.288 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.288 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.289 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.289 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-04-25T12:17:15.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-04-25T12:17:15.289 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.289 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.289 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.289 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.290 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.290 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-04-25T12:17:15.290 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.290 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.291 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.291 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.292 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.292 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.292 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.292 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.292 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.292 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.292 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.293 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.293 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.293 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.293 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.294 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.295 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.295 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.295 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.296 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.297 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.297 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.297 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.297 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.297 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.297 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.297 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.298 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.298 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.298 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.300 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.300 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.300 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.300 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-04-25T12:17:15.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-04-25T12:17:15.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.301 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.301 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.301 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-04-25T12:17:15.301 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.301 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.301 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T12:17:15.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.302 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-04-25T12:17:15.302 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-04-25T12:17:15.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.302 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-04-25T12:17:15.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.303 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T12:17:15.303 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.304 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-04-25T12:17:15.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-04-25T12:17:15.304 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-04-25T12:17:15.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.305 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.305 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.305 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-04-25T12:17:15.305 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.305 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.305 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-04-25T12:17:15.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-04-25T12:17:15.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.306 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.306 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T12:17:15.306 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.306 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T12:17:15.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.307 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.307 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.307 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-04-25T12:17:15.307 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-04-25T12:17:15.307 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.308 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.308 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.308 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.308 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T12:17:15.308 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.308 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.308 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.309 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:15.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:15.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.309 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:15.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.310 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T12:17:15.310 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.310 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.310 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-04-25T12:17:15.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-04-25T12:17:15.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.311 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.311 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.311 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-04-25T12:17:15.311 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.311 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T12:17:15.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.312 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.312 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.312 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.312 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.312 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-04-25T12:17:15.312 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.312 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.313 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.313 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.313 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.313 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.313 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.313 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.313 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.313 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.314 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.314 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.314 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.314 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.314 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.314 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.314 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.314 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.314 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.315 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.315 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.315 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.315 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.315 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.315 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.315 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.316 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.316 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.316 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.316 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.316 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.316 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.316 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.316 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.317 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.317 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.317 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.317 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.317 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.318 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.318 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.318 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.318 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.318 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.318 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.318 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.319 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.319 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.319 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.319 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.319 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.319 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.320 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.320 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.321 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-04-25T12:17:15.321 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-04-25T12:17:15.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.321 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.321 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T12:17:15.322 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.322 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.322 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.322 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.322 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.322 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.322 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.323 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-04-25T12:17:15.323 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-04-25T12:17:15.323 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.323 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-04-25T12:17:15.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.323 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.324 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.324 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.324 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.324 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.324 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:15.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:15.325 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.325 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.325 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.325 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:15.325 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.325 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.325 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.325 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.325 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.326 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.326 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.326 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.326 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-04-25T12:17:15.326 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-04-25T12:17:15.326 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-04-25T12:17:15.327 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.327 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-04-25T12:17:15.327 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-04-25T12:17:15.327 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.327 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-04-25T12:17:15.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.327 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.328 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.328 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T12:17:15.328 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.328 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.328 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.328 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-04-25T12:17:15.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-04-25T12:17:15.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.328 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.329 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.329 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-04-25T12:17:15.329 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.329 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.329 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.329 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T12:17:15.329 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.329 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.329 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.330 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-04-25T12:17:15.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-04-25T12:17:15.330 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.330 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.330 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.330 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.331 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.331 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-04-25T12:17:15.331 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.331 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.331 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.331 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:15.331 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:15.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.332 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-04-25T12:17:15.332 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-04-25T12:17:15.332 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.332 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.332 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T12:17:15.332 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.333 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.333 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.333 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.333 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.333 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.333 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.333 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.334 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.334 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.334 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.334 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.334 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.334 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.334 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.334 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.335 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.335 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.335 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.335 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.335 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.335 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.336 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.336 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.336 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.336 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.336 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.336 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.336 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.337 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.337 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.337 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.337 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.337 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.338 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.338 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.338 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.338 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.338 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.338 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.338 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.339 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.339 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.339 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.339 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.339 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.340 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.340 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.340 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.340 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.340 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.341 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.341 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.341 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.341 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.341 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-04-25T12:17:15.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-04-25T12:17:15.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.342 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.342 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.343 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.343 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.343 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-04-25T12:17:15.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.343 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.344 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.344 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-04-25T12:17:15.344 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-04-25T12:17:15.344 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.344 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.345 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-04-25T12:17:15.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.345 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.345 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.345 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-04-25T12:17:15.346 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.346 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.346 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.346 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T12:17:15.346 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T12:17:15.346 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.347 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.347 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.347 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-04-25T12:17:15.347 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.347 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.347 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.348 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T12:17:15.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.348 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.348 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-04-25T12:17:15.348 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-04-25T12:17:15.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.349 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T12:17:15.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T12:17:15.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-04-25T12:17:15.349 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T12:17:15.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.349 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.350 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-04-25T12:17:15.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-04-25T12:17:15.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.350 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.350 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.350 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-04-25T12:17:15.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.352 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-04-25T12:17:15.352 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-04-25T12:17:15.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.352 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-04-25T12:17:15.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.352 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.353 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T12:17:15.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.353 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.353 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.353 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-04-25T12:17:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.355 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.356 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.359 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.361 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-04-25T12:17:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.362 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.364 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.368 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-04-25T12:17:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.371 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-04-25T12:17:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-04-25T12:17:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-04-25T12:17:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.372 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T12:17:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-04-25T12:17:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-04-25T12:17:15.373 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-04-25T12:17:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T12:17:15.374 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.376 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.380 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.382 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-04-25T12:17:15.383 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-04-25T12:17:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-04-25T12:17:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-04-25T12:17:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-04-25T12:17:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T12:17:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.385 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.386 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.387 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.389 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.396 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.397 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.397 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-04-25T12:17:15.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-04-25T12:17:15.397 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-04-25T12:17:15.398 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T12:17:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-04-25T12:17:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-04-25T12:17:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.399 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-04-25T12:17:15.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.400 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-04-25T12:17:15.400 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.400 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.400 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-04-25T12:17:15.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-04-25T12:17:15.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-04-25T12:17:15.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T12:17:15.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.402 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-04-25T12:17:15.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-04-25T12:17:15.402 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.403 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.403 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.403 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.403 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-04-25T12:17:15.403 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.404 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.404 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-04-25T12:17:15.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-04-25T12:17:15.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.404 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.405 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-04-25T12:17:15.405 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.405 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.405 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.405 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T12:17:15.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T12:17:15.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.406 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.406 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.406 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-04-25T12:17:15.406 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.406 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.406 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.406 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.407 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.407 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.407 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.407 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.407 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.408 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.408 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.408 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.408 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.408 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.408 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.408 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.409 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.410 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.410 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.410 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.411 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.412 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.412 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.412 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.412 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.412 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.412 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.412 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.412 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.412 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.413 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.413 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.414 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.414 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.415 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.415 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.415 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-04-25T12:17:15.415 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.416 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.416 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.416 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.416 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.416 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.417 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.417 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.417 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.417 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.417 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.417 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.418 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.418 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.418 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.418 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.418 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.419 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.419 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.419 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.420 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.420 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.420 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.421 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.421 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.421 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.421 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.421 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.421 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.421 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.421 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.422 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.422 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.422 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.422 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.422 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.422 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.423 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.423 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.423 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.423 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.423 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.424 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.424 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.424 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.424 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-04-25T12:17:15.424 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.424 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.425 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.425 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.426 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.426 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.426 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.426 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.426 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.426 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.426 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.426 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.427 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.427 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.427 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.427 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.427 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.427 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.428 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.428 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.428 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.428 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.428 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.429 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.429 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.429 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.429 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.430 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.430 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.430 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.430 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.430 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.430 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.430 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.431 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.431 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.431 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.431 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.431 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.432 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.432 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.432 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.432 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.432 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.433 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.433 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T12:17:15.433 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.433 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.433 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.434 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-04-25T12:17:15.434 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-04-25T12:17:15.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.434 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.434 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.435 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.435 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T12:17:15.435 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.435 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.436 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.436 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.436 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.436 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.436 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.437 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.437 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.437 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.437 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.437 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.437 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.437 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.438 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.438 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.438 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.438 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.439 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.439 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.439 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.439 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.439 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.439 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.439 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.440 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.440 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.440 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.440 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.441 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.441 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.441 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.441 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.441 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.441 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.441 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.441 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.441 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.442 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.442 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.442 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.442 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.442 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.442 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.443 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.443 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.443 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.443 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-04-25T12:17:15.443 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T12:17:15.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.444 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.444 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.444 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.444 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-25T12:17:15.444 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-04-25T12:17:15.445 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-04-25T12:17:15.445 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.445 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-04-25T12:17:15.445 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.445 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.445 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.445 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.445 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-04-25T12:17:15.446 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.446 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.446 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.446 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.447 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.447 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.447 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.447 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.447 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.447 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.447 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.448 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.448 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.448 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.448 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.448 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.448 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.448 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.448 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.449 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.449 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.449 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.449 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.449 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.449 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.450 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.450 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.450 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.450 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.451 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.451 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.451 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.452 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.452 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.452 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.452 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.452 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.452 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.452 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.452 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.452 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.453 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.453 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.453 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.453 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.453 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.453 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.454 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.454 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.454 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.454 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-04-25T12:17:15.455 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.455 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.455 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.455 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.455 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.456 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.456 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.456 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.456 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.456 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.456 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.456 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.456 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.456 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.457 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.457 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.457 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.457 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.457 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.457 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.458 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.458 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.458 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.458 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.458 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.458 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.458 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.458 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.458 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.459 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.459 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.459 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.459 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.459 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.459 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.459 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.460 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.460 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.460 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.460 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.460 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.460 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.460 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.460 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.461 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.461 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.461 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.461 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.462 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.462 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.462 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.462 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.462 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.463 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.463 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.463 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-04-25T12:17:15.463 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.463 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.463 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.464 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.464 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.464 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.464 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.465 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.465 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.465 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.465 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.465 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.465 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.465 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.465 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.466 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.466 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.466 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.467 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.467 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.468 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.468 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.468 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.468 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.468 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.468 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.469 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.469 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.469 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.469 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.469 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.469 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.469 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.469 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.469 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.470 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.470 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.470 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.470 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.471 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.471 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.471 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.471 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.471 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:15.472 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:15.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:15.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.473 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.473 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-04-25T12:17:15.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-04-25T12:17:15.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:15.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.474 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.474 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T12:17:15.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-04-25T12:17:15.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-04-25T12:17:15.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T12:17:15.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.475 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.475 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.475 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.475 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-04-25T12:17:15.475 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.475 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.475 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.476 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-04-25T12:17:15.476 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-04-25T12:17:15.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.476 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.476 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.476 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.476 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.476 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.477 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T12:17:15.477 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.477 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.477 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.477 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T12:17:15.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T12:17:15.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.478 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.478 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.478 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.478 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.478 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.478 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.478 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.479 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.479 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.479 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.480 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.480 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.480 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.480 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.480 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.480 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.480 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.481 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.482 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.482 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.482 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.482 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.482 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.483 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.483 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.483 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.483 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.484 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.484 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.484 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.484 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.484 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.484 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.484 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.484 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.484 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.485 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.485 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.485 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.485 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.486 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.486 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.486 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.486 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.486 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.486 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.486 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.487 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-04-25T12:17:15.487 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-04-25T12:17:15.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.487 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.487 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.487 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-04-25T12:17:15.487 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.488 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.488 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.488 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-04-25T12:17:15.488 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.488 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.488 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.488 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:15.488 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:15.489 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.489 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.489 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.489 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T12:17:15.489 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.489 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.489 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.489 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-04-25T12:17:15.489 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.490 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.490 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.490 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.490 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.490 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-04-25T12:17:15.490 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-04-25T12:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-04-25T12:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.492 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.492 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.492 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.492 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.492 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.492 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-04-25T12:17:15.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-04-25T12:17:15.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T12:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T12:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.494 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.494 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.494 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.494 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.494 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.494 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.494 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.495 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.495 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.495 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.495 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.495 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.495 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.495 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.495 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.498 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-04-25T12:17:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-04-25T12:17:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.502 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.503 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T12:17:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T12:17:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-04-25T12:17:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-04-25T12:17:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-04-25T12:17:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:15.505 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-04-25T12:17:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.506 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T12:17:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T12:17:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T12:17:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T12:17:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.508 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T12:17:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T12:17:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-04-25T12:17:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.509 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T12:17:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-04-25T12:17:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-04-25T12:17:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.510 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-04-25T12:17:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T12:17:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T12:17:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-04-25T12:17:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.512 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-04-25T12:17:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-04-25T12:17:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.514 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T12:17:15.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T12:17:15.514 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.514 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T12:17:15.514 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T12:17:15.514 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T12:17:15.515 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.515 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.515 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.515 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.515 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.515 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-25T12:17:15.516 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.516 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.516 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.516 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.516 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.516 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.517 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.517 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.517 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.517 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.517 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.517 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.517 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.518 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.518 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.518 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.519 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.519 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.519 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.519 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.519 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.519 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.520 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.520 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.520 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.520 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.520 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.520 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.521 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.521 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.521 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.521 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.521 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.521 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.521 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.521 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.522 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.522 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.522 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.522 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.522 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.523 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.523 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.523 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.523 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.523 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.523 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.524 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.524 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.524 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.524 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-04-25T12:17:15.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-04-25T12:17:15.524 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.524 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:15.525 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.525 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.525 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.525 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.525 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.525 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.525 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.526 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-04-25T12:17:15.526 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T12:17:15.526 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.526 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.526 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.526 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.526 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.527 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.527 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.527 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.527 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.527 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.527 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-04-25T12:17:15.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-04-25T12:17:15.528 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.528 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.528 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.528 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.528 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.528 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.529 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.529 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.529 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.529 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.529 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.529 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.529 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-04-25T12:17:15.529 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.530 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.530 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.530 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-04-25T12:17:15.530 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-04-25T12:17:15.530 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.530 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.530 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.530 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.531 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.531 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.531 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.531 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.531 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.531 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.531 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T12:17:15.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T12:17:15.532 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.532 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.532 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.532 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.532 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.532 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.532 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.532 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.533 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.533 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.533 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.533 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.533 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.533 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.534 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.534 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.534 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.534 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.534 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.534 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.535 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.535 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.535 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.535 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.535 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.535 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.535 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.535 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.536 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.536 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.536 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.536 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.536 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.536 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.536 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.536 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.537 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.537 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.537 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.537 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.537 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.537 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.538 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.538 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.538 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.538 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.538 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.538 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.538 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.538 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.539 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.539 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.539 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.539 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.539 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.539 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.539 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.539 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.540 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.540 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.540 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.540 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.540 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.540 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.540 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.541 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.541 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.541 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.541 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.541 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T12:17:15.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T12:17:15.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.542 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.542 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.542 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.542 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.542 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.543 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.543 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.543 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T12:17:15.543 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T12:17:15.543 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.543 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.544 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.544 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.544 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.544 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.544 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.544 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.544 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.545 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.545 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.545 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.545 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-04-25T12:17:15.545 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.545 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.545 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.545 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.546 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.546 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.546 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.546 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.546 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.546 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.546 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.547 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.547 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.547 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.547 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.547 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.547 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.548 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.548 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.548 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.549 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.549 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.549 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.549 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.549 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.549 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.550 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.550 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.550 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.550 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.550 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.550 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.550 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.550 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.551 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.551 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.551 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.551 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.551 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.551 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.551 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.552 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.552 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.552 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.552 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.552 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.552 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.552 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.552 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.553 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.553 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.553 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.553 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.553 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.553 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.553 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.554 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.554 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.554 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.554 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.554 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-04-25T12:17:15.554 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.555 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.555 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.555 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.555 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.555 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.555 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.555 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.556 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.556 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.556 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.556 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.556 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.556 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.556 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.556 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.556 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.557 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.557 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.557 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.557 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.557 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.557 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.558 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.558 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.558 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.558 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.558 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.558 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.558 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.559 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.559 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.559 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.559 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.559 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.559 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.559 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.560 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.560 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.560 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.560 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.560 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.560 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.560 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.560 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.560 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.561 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.561 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.561 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.561 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.561 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.561 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.561 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.561 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.562 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.562 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.562 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.562 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.562 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.562 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.562 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.563 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.563 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.563 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.563 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-04-25T12:17:15.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-04-25T12:17:15.563 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.563 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.563 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.564 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-04-25T12:17:15.564 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.564 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.564 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.564 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T12:17:15.564 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.564 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.564 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.564 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.565 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.565 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.565 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-04-25T12:17:15.565 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.565 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.565 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.565 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.565 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.566 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.566 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.566 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.566 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.566 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.566 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.566 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.566 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.566 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.569 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.569 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.569 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.569 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.569 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.569 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.569 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.570 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.570 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.570 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.570 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.570 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.570 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.570 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.570 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.571 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.571 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.571 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.571 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.571 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.571 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.571 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.571 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.572 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.572 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.572 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.572 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.572 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.572 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.572 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.573 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.573 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.573 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.573 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-04-25T12:17:15.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-04-25T12:17:15.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.574 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.574 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.574 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.574 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.574 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.574 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.574 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.575 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-04-25T12:17:15.575 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-04-25T12:17:15.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.575 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.575 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-04-25T12:17:15.575 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.575 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.576 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.576 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.576 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-04-25T12:17:15.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-04-25T12:17:15.576 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.577 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.577 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.577 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.577 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.578 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.578 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-04-25T12:17:15.578 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-04-25T12:17:15.578 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.578 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.578 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.578 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.579 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.579 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.579 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.579 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.579 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.579 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.579 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.579 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-04-25T12:17:15.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-04-25T12:17:15.580 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:15.580 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.580 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.580 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.580 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.580 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.580 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.581 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.581 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.581 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-04-25T12:17:15.581 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-04-25T12:17:15.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.582 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.582 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.582 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.582 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.582 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.582 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.582 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.582 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-04-25T12:17:15.583 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-04-25T12:17:15.583 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.583 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.583 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.583 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.583 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.583 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.583 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.584 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.584 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.584 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-04-25T12:17:15.584 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-04-25T12:17:15.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.584 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.584 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.585 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.585 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.585 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.585 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.585 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.585 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-04-25T12:17:15.586 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-04-25T12:17:15.586 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.586 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.586 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.586 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.586 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T12:17:15.586 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-04-25T12:17:15.586 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.587 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.587 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.587 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.587 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.587 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.587 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-04-25T12:17:15.587 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-04-25T12:17:15.587 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.587 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.588 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.588 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.588 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.588 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.588 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.588 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.588 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.588 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.588 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.589 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-04-25T12:17:15.589 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-04-25T12:17:15.589 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.589 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.589 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.589 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.589 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.590 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.590 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.590 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.590 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.590 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.590 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-04-25T12:17:15.590 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-04-25T12:17:15.591 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.591 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.591 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.591 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.591 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.591 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.591 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.591 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.591 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.592 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.592 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.592 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.592 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.592 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T12:17:15.592 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.592 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.592 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.593 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-04-25T12:17:15.593 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-04-25T12:17:15.593 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.593 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.593 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.593 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.593 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.593 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.593 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.594 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T12:17:15.594 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.594 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.594 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.594 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-04-25T12:17:15.594 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-04-25T12:17:15.594 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.595 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.595 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.595 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.595 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.595 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.595 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-04-25T12:17:15.595 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.596 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.596 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.596 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.596 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.596 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.596 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.596 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.596 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.597 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.597 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.597 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.597 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.597 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.597 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.597 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.598 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.598 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.598 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.598 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.598 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.598 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.599 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.599 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.599 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.599 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.599 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.599 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.599 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.599 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.599 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.600 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.600 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.600 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.600 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.600 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.600 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.600 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.600 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.601 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.601 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.601 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.601 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.601 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.601 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.601 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.601 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.601 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.602 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.602 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.602 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.602 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.602 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.602 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.602 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.603 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.603 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.603 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.603 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.603 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.603 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.604 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.604 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.604 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.604 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.604 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.604 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-04-25T12:17:15.604 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.604 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.605 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.605 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.605 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.605 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.605 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.605 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.605 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.605 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.605 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.606 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.606 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.606 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.606 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.606 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.606 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.606 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.606 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.606 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.607 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.607 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.607 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.607 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.607 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.607 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.607 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.608 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.608 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.608 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.608 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.608 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.608 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.608 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.608 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.608 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.609 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.609 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.609 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.609 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.609 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.609 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.609 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.609 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.609 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.609 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.610 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.610 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.610 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.610 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.610 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.610 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.610 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.610 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.610 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.611 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.611 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.611 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.611 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.611 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.611 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.612 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.612 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.612 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.612 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.612 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.612 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.612 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.612 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.613 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-04-25T12:17:15.613 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.613 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.613 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.613 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.614 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.614 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.614 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.614 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.614 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.614 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.614 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.615 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.615 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.615 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.615 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.615 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.615 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.615 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.615 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.616 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.616 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.616 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.616 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.616 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.616 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.616 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.617 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.617 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.617 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.617 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.617 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.617 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.617 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.617 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.618 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.618 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.618 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.618 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.618 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.618 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.618 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.618 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.619 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.619 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.619 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.619 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.619 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.619 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.619 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.619 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.619 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.620 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.620 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.620 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.620 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.620 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.620 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.620 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.621 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.621 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.621 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.621 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.621 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.621 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.621 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.621 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.622 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.622 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-04-25T12:17:15.622 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.622 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.622 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.623 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-04-25T12:17:15.623 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-04-25T12:17:15.623 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.623 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.623 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.623 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T12:17:15.623 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.623 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.624 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.624 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.624 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.624 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.624 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.624 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.624 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:15.625 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:15.625 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.625 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.626 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-04-25T12:17:15.626 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.626 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.627 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.627 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.627 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.627 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.627 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.627 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.628 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.628 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.628 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.628 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.628 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.628 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.629 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.629 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.629 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.629 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.629 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.629 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.629 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.629 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.630 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.630 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.630 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.630 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.630 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.630 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.630 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.631 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.631 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.631 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.631 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.631 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.631 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.632 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.632 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.632 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.632 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.632 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.632 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.632 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.632 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.633 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.633 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.633 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.633 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.633 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.633 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.633 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.634 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.634 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.634 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.634 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.634 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.634 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.635 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.635 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.635 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.635 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.635 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.635 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.635 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.635 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.636 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.636 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.636 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.636 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.636 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-04-25T12:17:15.636 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.636 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.636 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.637 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-04-25T12:17:15.637 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-04-25T12:17:15.637 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.637 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.637 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.637 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.637 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.637 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T12:17:15.638 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.638 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.638 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.638 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-04-25T12:17:15.638 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-04-25T12:17:15.639 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.639 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.639 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.639 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.639 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.639 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.639 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T12:17:15.639 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.640 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.640 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.640 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-04-25T12:17:15.640 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-04-25T12:17:15.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.640 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.640 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.640 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.641 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.641 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.641 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.641 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T12:17:15.641 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.641 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.641 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.641 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-04-25T12:17:15.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-04-25T12:17:15.642 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.642 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.642 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.642 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.642 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.642 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.642 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.642 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T12:17:15.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.643 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.643 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.643 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-04-25T12:17:15.643 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-04-25T12:17:15.643 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.643 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.643 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.644 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.644 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.644 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.644 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.644 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T12:17:15.644 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.644 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.644 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.645 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-04-25T12:17:15.645 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-04-25T12:17:15.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.645 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.645 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.645 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.646 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.646 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.646 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.646 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.646 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.646 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-04-25T12:17:15.646 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-04-25T12:17:15.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.647 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.647 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.647 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.647 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.647 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.647 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.647 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.647 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.648 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.648 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.648 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-04-25T12:17:15.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-04-25T12:17:15.648 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.648 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.648 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.649 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T12:17:15.649 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.649 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.649 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.649 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.649 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.649 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.649 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T12:17:15.650 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T12:17:15.650 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.650 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.650 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.650 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.650 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.651 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.651 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.651 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.651 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:15.651 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:15.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.651 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.652 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.652 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-04-25T12:17:15.652 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-04-25T12:17:15.652 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.652 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.652 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.652 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.652 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.652 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.653 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-04-25T12:17:15.653 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-04-25T12:17:15.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.653 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.653 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.653 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-04-25T12:17:15.653 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.653 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.654 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.654 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.654 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.654 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.654 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-04-25T12:17:15.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-04-25T12:17:15.655 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.655 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.655 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.655 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.655 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.656 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.656 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.656 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.656 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.656 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-04-25T12:17:15.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-04-25T12:17:15.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.656 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.657 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.657 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.657 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.657 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.657 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.657 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.657 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.657 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.658 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.658 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.658 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.658 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.658 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.658 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.658 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.659 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.659 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.659 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.659 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.659 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.659 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.659 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.659 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.660 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.660 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.660 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.660 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.660 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.660 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.660 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.660 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.661 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.661 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.661 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.661 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.661 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.661 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.661 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.661 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.661 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.662 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.662 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.662 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.662 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.662 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.662 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.662 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.663 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.663 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.663 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.663 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.663 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.665 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.666 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.666 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.667 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.667 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.667 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.667 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.667 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.667 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.668 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.668 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.668 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.668 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.668 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.668 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.668 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.668 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.668 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.669 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.669 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.669 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-04-25T12:17:15.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-04-25T12:17:15.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.669 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.669 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.669 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.670 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.670 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.670 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.670 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.670 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.670 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-04-25T12:17:15.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-04-25T12:17:15.671 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.671 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.671 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.671 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.671 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.671 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.671 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.671 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.671 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.672 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.672 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.672 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-04-25T12:17:15.672 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-04-25T12:17:15.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.672 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.672 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.673 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T12:17:15.673 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.673 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.673 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.673 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.673 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.673 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.673 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.673 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.674 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.674 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.674 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-04-25T12:17:15.674 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.674 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.674 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.674 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.674 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.675 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.675 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.675 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.675 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.675 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.675 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.675 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.675 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.676 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.676 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.676 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.676 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.676 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.676 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.676 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.676 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.676 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.677 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.677 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.677 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.677 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.677 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.677 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.677 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.677 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.678 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.678 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.678 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.678 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.678 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.678 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.679 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.679 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.679 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.679 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.679 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.679 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.679 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.679 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.680 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.680 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.680 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.680 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.680 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.680 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.680 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.680 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.680 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.681 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.681 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.681 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.681 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.681 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.681 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.681 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.681 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.682 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.682 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.682 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.682 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.682 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.682 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.682 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.682 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.683 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.683 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-25T12:17:15.683 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.683 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.683 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.683 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.683 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.683 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.683 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.684 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.684 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.684 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.684 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.684 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.684 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.684 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.684 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.685 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.685 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.685 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.685 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.685 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.685 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.685 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.685 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.685 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.686 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.686 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.686 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.686 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.686 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.686 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.686 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.686 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.687 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.687 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.687 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.687 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.687 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.687 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.687 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.687 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.688 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.688 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.688 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.688 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.688 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.688 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.688 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.688 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.688 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.688 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.689 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.689 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.689 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.689 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.689 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.689 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.689 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.690 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.690 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.690 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.690 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.690 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.690 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.690 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.690 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.691 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.691 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.691 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.691 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-04-25T12:17:15.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-04-25T12:17:15.691 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.691 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.691 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.692 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-04-25T12:17:15.692 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.692 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.692 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T12:17:15.692 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.692 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.692 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.693 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-04-25T12:17:15.693 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-04-25T12:17:15.693 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.693 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.693 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.693 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.693 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.693 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.693 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.694 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-04-25T12:17:15.694 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.694 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.694 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.694 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-04-25T12:17:15.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-04-25T12:17:15.694 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.695 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.695 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.695 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:15.695 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.695 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.695 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.695 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T12:17:15.695 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.696 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.696 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.696 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-04-25T12:17:15.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T12:17:15.696 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.696 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.696 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.696 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.697 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.697 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.697 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.697 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T12:17:15.697 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.697 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.697 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.697 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.698 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.698 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-04-25T12:17:15.698 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.698 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.698 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.698 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-04-25T12:17:15.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-04-25T12:17:15.699 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:15.699 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.699 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-04-25T12:17:15.699 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.699 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.699 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.699 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.700 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.700 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.700 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-04-25T12:17:15.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-04-25T12:17:15.700 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.700 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.700 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.701 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.701 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.701 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.701 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.701 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.701 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.701 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.701 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.702 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:15.702 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:15.702 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.702 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.702 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.702 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:15.702 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.702 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.703 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.703 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.703 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.703 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.703 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.703 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.703 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.704 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.704 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.704 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.704 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.704 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.704 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.704 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.704 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.704 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.705 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.705 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.705 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.705 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.705 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.705 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.705 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.705 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.705 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.706 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.706 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.706 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.706 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.706 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.706 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.707 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.707 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.707 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.707 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.707 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.707 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.708 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.708 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.708 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.708 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.708 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.708 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.708 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.709 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.709 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.709 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.709 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.709 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.709 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.709 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.710 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.710 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.710 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.710 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.710 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.710 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.710 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.710 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.711 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.711 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.711 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.711 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.711 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.711 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.711 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.712 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-04-25T12:17:15.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-04-25T12:17:15.712 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.712 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.712 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.712 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.712 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.713 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.713 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.713 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.713 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-04-25T12:17:15.714 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-04-25T12:17:15.714 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.714 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.714 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.714 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-04-25T12:17:15.714 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.714 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.714 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.715 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.715 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.715 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.715 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-04-25T12:17:15.715 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-04-25T12:17:15.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.715 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.716 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.716 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T12:17:15.716 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.716 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.716 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.716 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.716 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.716 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.716 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.717 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T12:17:15.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.717 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.717 INFO:teuthology.orchestra.run.smithi019.stdout: "modules": [ 2024-04-25T12:17:15.717 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-25T12:17:15.717 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard", 2024-04-25T12:17:15.718 INFO:teuthology.orchestra.run.smithi019.stdout: "iostat", 2024-04-25T12:17:15.718 INFO:teuthology.orchestra.run.smithi019.stdout: "nfs", 2024-04-25T12:17:15.718 INFO:teuthology.orchestra.run.smithi019.stdout: "restful", 2024-04-25T12:17:15.718 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_schedule" 2024-04-25T12:17:15.718 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.718 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-04-25T12:17:15.718 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.719 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-04-25T12:17:15.719 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.719 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.719 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.719 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:15.719 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:15.719 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.719 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.719 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.720 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:15.720 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.720 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.720 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.720 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T12:17:15.720 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.720 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.721 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.721 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.721 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.721 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.721 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.721 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.721 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.721 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.721 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.721 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.722 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.722 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.722 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.722 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.722 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.722 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.722 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.722 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.722 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.723 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.723 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.723 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.723 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.723 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.723 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.723 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.723 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.724 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.724 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.724 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.724 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.724 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.724 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.724 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.724 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.725 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.725 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.725 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.725 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.725 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.725 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.725 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.725 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.726 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.726 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.726 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.726 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.726 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.726 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.726 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.726 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.727 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.727 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.727 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.727 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.727 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.727 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.727 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.728 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.728 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.728 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.728 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.728 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.728 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.728 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.728 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.729 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-04-25T12:17:15.729 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-04-25T12:17:15.729 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.729 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.729 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.729 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.729 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.729 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.730 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.730 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-04-25T12:17:15.730 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.730 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.730 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.730 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-04-25T12:17:15.730 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-04-25T12:17:15.730 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.731 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-04-25T12:17:15.731 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.731 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.731 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.731 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-04-25T12:17:15.731 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.731 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.732 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.732 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-04-25T12:17:15.732 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-04-25T12:17:15.732 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.732 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.732 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.732 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.732 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.732 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.733 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-04-25T12:17:15.733 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.733 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.733 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-04-25T12:17:15.733 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-04-25T12:17:15.733 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.734 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.734 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.734 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.734 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.734 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.734 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.734 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-04-25T12:17:15.734 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.735 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.735 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.735 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-04-25T12:17:15.735 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-04-25T12:17:15.735 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.735 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.735 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.735 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-04-25T12:17:15.736 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.736 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.736 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.736 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-04-25T12:17:15.736 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.736 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.736 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.736 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-04-25T12:17:15.737 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-04-25T12:17:15.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.737 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.737 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.737 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.737 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-04-25T12:17:15.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.738 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.738 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-04-25T12:17:15.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-04-25T12:17:15.738 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.738 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.738 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.739 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.739 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.739 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.739 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T12:17:15.739 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.739 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.739 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.740 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-04-25T12:17:15.740 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-04-25T12:17:15.740 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.740 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.740 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.740 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.740 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.740 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.740 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.741 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-04-25T12:17:15.741 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.741 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.741 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.741 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.741 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.741 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.741 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-04-25T12:17:15.742 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.742 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.742 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.742 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-04-25T12:17:15.742 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-04-25T12:17:15.742 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.742 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.742 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.742 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.743 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.743 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.743 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.743 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T12:17:15.743 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.743 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.743 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.744 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-04-25T12:17:15.744 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-04-25T12:17:15.744 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.744 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.744 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.744 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-04-25T12:17:15.744 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.744 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.744 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.745 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T12:17:15.745 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T12:17:15.745 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.745 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.745 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-04-25T12:17:15.745 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-04-25T12:17:15.745 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:15.746 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.746 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.746 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:15.746 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T12:17:15.746 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T12:17:15.746 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.746 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T12:17:15.746 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T12:17:15.746 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.747 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.747 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-04-25T12:17:15.747 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-04-25T12:17:15.747 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:15.747 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.747 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.748 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-04-25T12:17:15.748 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T12:17:15.748 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-04-25T12:17:15.748 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.748 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T12:17:15.748 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.748 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.748 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.749 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-04-25T12:17:15.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-04-25T12:17:15.749 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.749 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.749 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.750 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T12:17:15.750 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.750 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.750 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.750 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T12:17:15.750 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T12:17:15.750 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.750 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.751 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-04-25T12:17:15.751 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-04-25T12:17:15.751 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:15.751 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.751 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.751 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-04-25T12:17:15.751 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-04-25T12:17:15.752 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-04-25T12:17:15.752 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.752 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-04-25T12:17:15.752 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T12:17:15.752 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.752 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.752 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-04-25T12:17:15.753 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-04-25T12:17:15.753 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.753 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.753 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.753 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-04-25T12:17:15.753 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.753 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.753 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.754 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T12:17:15.754 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T12:17:15.754 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.754 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.754 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-04-25T12:17:15.754 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-04-25T12:17:15.755 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:15.755 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.755 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.755 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:15.755 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-04-25T12:17:15.755 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-04-25T12:17:15.755 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.755 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T12:17:15.756 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T12:17:15.756 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.756 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.756 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.756 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.756 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.756 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.756 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.757 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.757 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.757 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.757 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.757 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.757 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.757 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.757 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.758 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.758 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.758 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.758 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.758 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.758 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.758 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.759 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.759 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.759 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.759 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.759 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.759 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.759 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.759 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.760 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.760 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.760 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.760 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.760 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.760 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.760 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.761 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.761 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.761 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.761 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.761 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.761 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.762 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.762 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.762 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.762 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.762 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.762 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.762 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.762 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.763 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.764 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.764 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.764 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.764 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.764 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.764 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.764 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.764 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.765 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-04-25T12:17:15.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-04-25T12:17:15.765 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:15.765 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.765 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.765 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:15.765 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.766 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T12:17:15.766 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.766 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.766 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.766 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T12:17:15.767 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T12:17:15.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.767 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-04-25T12:17:15.767 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.767 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.767 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.768 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-04-25T12:17:15.768 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-04-25T12:17:15.768 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-04-25T12:17:15.768 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-04-25T12:17:15.768 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-04-25T12:17:15.768 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.768 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-04-25T12:17:15.768 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.769 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.769 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.769 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-04-25T12:17:15.769 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-04-25T12:17:15.769 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.769 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.769 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.770 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.770 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.770 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.770 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.770 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T12:17:15.770 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.770 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.770 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.771 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:15.771 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:15.771 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.771 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.771 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:15.771 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.772 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.772 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.772 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T12:17:15.772 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.772 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.772 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.772 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-04-25T12:17:15.772 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-04-25T12:17:15.773 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.773 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.773 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.773 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T12:17:15.773 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T12:17:15.773 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.773 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.773 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T12:17:15.773 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T12:17:15.774 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.774 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.774 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-04-25T12:17:15.774 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-04-25T12:17:15.774 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:15.774 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.774 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.774 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:15.775 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.775 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.775 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.775 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T12:17:15.775 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.775 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.775 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.775 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.775 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.776 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-04-25T12:17:15.776 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.776 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.776 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.776 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-04-25T12:17:15.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-04-25T12:17:15.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:15.777 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.777 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.777 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T12:17:15.777 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.777 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.777 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.777 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T12:17:15.777 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.777 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.778 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.778 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-04-25T12:17:15.778 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-04-25T12:17:15.778 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.778 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.778 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.778 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-04-25T12:17:15.779 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.779 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.779 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.779 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T12:17:15.779 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.779 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.779 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-04-25T12:17:15.780 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-04-25T12:17:15.780 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.780 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.780 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.780 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-04-25T12:17:15.780 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.780 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.780 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T12:17:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-04-25T12:17:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-04-25T12:17:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.781 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.782 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.782 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.782 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.782 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T12:17:15.782 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T12:17:15.782 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.782 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.783 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-04-25T12:17:15.783 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-04-25T12:17:15.783 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.783 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.783 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.783 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:15.783 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.783 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.783 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.784 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T12:17:15.784 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.784 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.784 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.784 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-04-25T12:17:15.784 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T12:17:15.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:15.784 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.785 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.785 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-04-25T12:17:15.785 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.785 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.785 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.785 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T12:17:15.785 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.785 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.786 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-04-25T12:17:15.786 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-04-25T12:17:15.786 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.786 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.786 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.786 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.786 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.786 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.787 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.787 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-04-25T12:17:15.787 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-04-25T12:17:15.787 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-04-25T12:17:15.787 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.787 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T12:17:15.787 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.787 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.787 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.788 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-04-25T12:17:15.788 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-04-25T12:17:15.788 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.788 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.788 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.788 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.788 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.789 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.789 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.789 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T12:17:15.789 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.789 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.789 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.789 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-04-25T12:17:15.789 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-04-25T12:17:15.790 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.790 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.790 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.790 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.790 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.790 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.790 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.790 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T12:17:15.790 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.791 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.791 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.791 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-04-25T12:17:15.791 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-04-25T12:17:15.791 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.791 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.791 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.792 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.792 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.792 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.792 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T12:17:15.792 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.792 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.792 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.792 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-04-25T12:17:15.793 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-04-25T12:17:15.793 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.793 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.793 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.793 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T12:17:15.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.793 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.793 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.793 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T12:17:15.794 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.794 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.794 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.794 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-04-25T12:17:15.794 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-04-25T12:17:15.794 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.794 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.795 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T12:17:15.795 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.795 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.795 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.795 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-04-25T12:17:15.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.795 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.795 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.796 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-04-25T12:17:15.796 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-04-25T12:17:15.796 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.796 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.796 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.796 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T12:17:15.796 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.796 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.796 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-04-25T12:17:15.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.797 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.797 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.797 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-04-25T12:17:15.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-04-25T12:17:15.797 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.797 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.798 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.798 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T12:17:15.798 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.798 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.798 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.798 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T12:17:15.798 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.798 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.798 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.799 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-04-25T12:17:15.799 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-04-25T12:17:15.799 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.799 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.799 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T12:17:15.799 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.799 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.800 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.800 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-04-25T12:17:15.800 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.800 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.800 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.800 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-04-25T12:17:15.800 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-04-25T12:17:15.800 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.801 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.801 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.801 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T12:17:15.801 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.801 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.801 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.801 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-04-25T12:17:15.801 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.801 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.802 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.802 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-04-25T12:17:15.802 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-04-25T12:17:15.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.802 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.802 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.802 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T12:17:15.802 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.803 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.803 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.803 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-04-25T12:17:15.803 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.803 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.803 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.804 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-04-25T12:17:15.804 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-04-25T12:17:15.804 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.804 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.804 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.804 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T12:17:15.804 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.804 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.804 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.805 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-04-25T12:17:15.805 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.805 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.805 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-04-25T12:17:15.805 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-04-25T12:17:15.805 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.805 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.806 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T12:17:15.806 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.806 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.806 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-04-25T12:17:15.806 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.807 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.807 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.807 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-04-25T12:17:15.807 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-04-25T12:17:15.807 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.807 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.807 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.807 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T12:17:15.807 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.808 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.808 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-04-25T12:17:15.808 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.808 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.808 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.808 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.808 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-04-25T12:17:15.809 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-04-25T12:17:15.809 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.809 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.809 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T12:17:15.809 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.809 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.809 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.809 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T12:17:15.810 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.810 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.810 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.810 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-04-25T12:17:15.810 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-04-25T12:17:15.810 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.810 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.810 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.811 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T12:17:15.811 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.811 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.811 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.811 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-04-25T12:17:15.811 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.811 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.811 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.812 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-04-25T12:17:15.812 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-04-25T12:17:15.812 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.812 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.812 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.812 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T12:17:15.812 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.813 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.813 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-04-25T12:17:15.813 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.813 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.813 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.813 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-04-25T12:17:15.813 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-04-25T12:17:15.814 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.814 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.814 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.814 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T12:17:15.814 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.814 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.814 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.814 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-04-25T12:17:15.815 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.815 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.815 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.815 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-04-25T12:17:15.815 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-04-25T12:17:15.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.815 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.815 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.816 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T12:17:15.816 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.816 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.816 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-04-25T12:17:15.816 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.816 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.816 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.816 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-04-25T12:17:15.817 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-04-25T12:17:15.817 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.817 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.817 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.817 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T12:17:15.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.817 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.817 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.817 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-04-25T12:17:15.818 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.818 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.818 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.818 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-04-25T12:17:15.818 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-04-25T12:17:15.818 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.818 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.818 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.819 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.819 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.819 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.819 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.819 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T12:17:15.819 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.819 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.819 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.820 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-04-25T12:17:15.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-04-25T12:17:15.820 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.820 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.820 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.820 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:15.820 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.820 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.821 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.821 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T12:17:15.821 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.821 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.821 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.821 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-04-25T12:17:15.821 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T12:17:15.822 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.822 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.822 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.822 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-04-25T12:17:15.822 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.822 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.822 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.822 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T12:17:15.822 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.823 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.823 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-04-25T12:17:15.823 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-04-25T12:17:15.823 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.823 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.823 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.824 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-04-25T12:17:15.824 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.824 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.824 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.824 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T12:17:15.824 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.824 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.824 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.825 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-04-25T12:17:15.825 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-04-25T12:17:15.825 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.825 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.825 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.825 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-04-25T12:17:15.825 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.826 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.826 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.826 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-04-25T12:17:15.826 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.826 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.826 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.826 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-04-25T12:17:15.826 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-04-25T12:17:15.827 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.827 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.827 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.827 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.827 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.827 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.827 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.827 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T12:17:15.827 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.828 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.828 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.828 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.828 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-04-25T12:17:15.828 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-04-25T12:17:15.828 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.828 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.828 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.828 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:15.829 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.829 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.829 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.829 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-04-25T12:17:15.829 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.829 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.829 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.829 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-04-25T12:17:15.829 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-04-25T12:17:15.830 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.830 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.830 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.830 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T12:17:15.830 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.830 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.830 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.830 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T12:17:15.830 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.831 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.831 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.831 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-04-25T12:17:15.831 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-04-25T12:17:15.831 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.831 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.831 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.832 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:15.832 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.832 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.832 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.832 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-04-25T12:17:15.832 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.832 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.832 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.833 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-04-25T12:17:15.833 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-04-25T12:17:15.833 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.833 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.833 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.833 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.833 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.833 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.834 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.834 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T12:17:15.834 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.834 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.834 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.834 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-04-25T12:17:15.834 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-04-25T12:17:15.835 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.835 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.835 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.835 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.835 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.835 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.835 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.835 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T12:17:15.835 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.836 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.836 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.836 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.836 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.836 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.836 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.836 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.836 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.837 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.837 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.837 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.837 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.837 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.837 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.837 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.837 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.837 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.838 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.838 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.838 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.838 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.838 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.838 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-04-25T12:17:15.838 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-04-25T12:17:15.838 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.839 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.839 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.839 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.839 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.839 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.839 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.839 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T12:17:15.839 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.839 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.840 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.840 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.840 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.840 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.840 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.840 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.840 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.840 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.841 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.841 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.841 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T12:17:15.841 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.841 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.841 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.841 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.841 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.841 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.841 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.842 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.842 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.842 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.842 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.842 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.842 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.842 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.842 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.842 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.843 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.843 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.843 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.843 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.843 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.843 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.843 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.843 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.844 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.844 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.844 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.844 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.844 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.844 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.844 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.844 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.845 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.845 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.845 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.845 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.845 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.845 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T12:17:15.845 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T12:17:15.845 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.846 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.846 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.846 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.846 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.846 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.846 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.846 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T12:17:15.846 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.847 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.847 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.847 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T12:17:15.847 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T12:17:15.847 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.847 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.847 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.847 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-04-25T12:17:15.847 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.848 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.848 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.848 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T12:17:15.848 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.848 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.848 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.848 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-04-25T12:17:15.849 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-04-25T12:17:15.849 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.849 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.849 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.849 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:15.849 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.849 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.849 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.850 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-04-25T12:17:15.850 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.850 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.850 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.850 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.850 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-04-25T12:17:15.850 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-04-25T12:17:15.850 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.850 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.851 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.851 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:15.851 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.851 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.851 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.851 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T12:17:15.851 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.851 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.851 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.852 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-04-25T12:17:15.852 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-04-25T12:17:15.852 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.852 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.852 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.852 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.852 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.852 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.853 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.853 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-04-25T12:17:15.853 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.853 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.853 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.853 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.853 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-04-25T12:17:15.853 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-04-25T12:17:15.854 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.854 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.854 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.854 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-04-25T12:17:15.854 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.854 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.854 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.854 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-04-25T12:17:15.855 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-04-25T12:17:15.855 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.855 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T12:17:15.855 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.855 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.855 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.855 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.855 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-04-25T12:17:15.856 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-04-25T12:17:15.856 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.856 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.856 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.856 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-04-25T12:17:15.856 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.856 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.856 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.856 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T12:17:15.857 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.857 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.857 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.857 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.857 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-04-25T12:17:15.857 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-04-25T12:17:15.857 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.857 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.858 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.858 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T12:17:15.858 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.858 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.858 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.858 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T12:17:15.858 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.858 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.859 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.859 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.859 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-04-25T12:17:15.859 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-04-25T12:17:15.859 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.859 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.859 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.859 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.860 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.860 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.860 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.860 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T12:17:15.860 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.860 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.860 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.860 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.860 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-04-25T12:17:15.861 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-04-25T12:17:15.861 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.861 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.861 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.861 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.861 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.861 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.861 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.862 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T12:17:15.862 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.862 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.862 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.862 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.862 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-04-25T12:17:15.862 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-04-25T12:17:15.862 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.862 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.863 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.863 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.863 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.863 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.863 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.863 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T12:17:15.863 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.863 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.863 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.864 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-04-25T12:17:15.864 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-04-25T12:17:15.864 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.864 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.864 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.864 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.865 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.865 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.865 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.865 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T12:17:15.865 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.865 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.865 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.865 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.865 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-04-25T12:17:15.866 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-04-25T12:17:15.866 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.866 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.866 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.866 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.870 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.870 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.870 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.871 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T12:17:15.871 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.871 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.871 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.871 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-04-25T12:17:15.872 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-04-25T12:17:15.872 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.872 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.872 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.872 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-04-25T12:17:15.872 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.872 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.872 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.872 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-04-25T12:17:15.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.873 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.873 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.873 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-04-25T12:17:15.873 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-04-25T12:17:15.873 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.873 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.874 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.874 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.874 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.874 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.874 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T12:17:15.874 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.874 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.875 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.875 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-04-25T12:17:15.875 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-04-25T12:17:15.875 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.875 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.875 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.875 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.876 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.876 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.876 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.876 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T12:17:15.876 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.876 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.876 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.876 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-04-25T12:17:15.876 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-04-25T12:17:15.877 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.877 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.877 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.877 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.877 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.877 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.877 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.877 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T12:17:15.877 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.878 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.878 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.878 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-04-25T12:17:15.878 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-04-25T12:17:15.878 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.878 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.878 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.878 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.879 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.879 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.879 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.879 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T12:17:15.879 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.879 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.879 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.879 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.879 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-04-25T12:17:15.880 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-04-25T12:17:15.880 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.880 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.880 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.880 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.880 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.880 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.880 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.880 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T12:17:15.881 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.881 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.881 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.881 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.881 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-04-25T12:17:15.881 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-04-25T12:17:15.881 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.881 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.881 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.881 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.882 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.882 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.882 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.882 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T12:17:15.882 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.882 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.882 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.882 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.883 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.883 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.883 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-04-25T12:17:15.883 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.883 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.883 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.883 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.884 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.884 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.884 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.884 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.884 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.884 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.884 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.885 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.885 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.885 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.885 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.885 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.885 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.885 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.885 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.885 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.886 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.886 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.886 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.886 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.886 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.886 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.886 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.886 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.887 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.887 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.887 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.887 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.887 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.887 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.887 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.887 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.888 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.888 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.888 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.888 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.888 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.888 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.888 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.888 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.889 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.889 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.889 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.889 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.889 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.889 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.889 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.889 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.890 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.890 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.890 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.890 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.890 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.890 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.890 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.891 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.891 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.891 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.891 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.891 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.891 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.891 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.891 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.891 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.892 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-04-25T12:17:15.892 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-04-25T12:17:15.892 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.892 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.892 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.892 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-04-25T12:17:15.892 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.892 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.893 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.893 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T12:17:15.893 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.893 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.893 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.893 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-04-25T12:17:15.894 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-04-25T12:17:15.894 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:15.894 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.894 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.894 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-04-25T12:17:15.894 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.894 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.895 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.895 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T12:17:15.895 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.895 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.895 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.895 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.895 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:15.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.895 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:15.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-04-25T12:17:15.896 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:15.896 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:15.896 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:15.896 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T12:17:15.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T12:17:15.896 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.896 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.896 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.897 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:15.897 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.897 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.897 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.897 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.897 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.897 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.897 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.898 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T12:17:15.898 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T12:17:15.898 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.898 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.898 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.898 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.898 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.898 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.899 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.899 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.899 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.899 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.899 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.899 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.899 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T12:17:15.899 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T12:17:15.899 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.900 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.900 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.900 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.900 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.900 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.900 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.900 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.900 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.901 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.901 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.901 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-04-25T12:17:15.901 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T12:17:15.901 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.901 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.901 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.902 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.902 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.902 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.902 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.902 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.902 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.902 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.902 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T12:17:15.903 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T12:17:15.903 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.903 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.903 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.903 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.903 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.904 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.904 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.904 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.904 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.904 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T12:17:15.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T12:17:15.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.904 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.905 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.905 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.905 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.905 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.905 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.905 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.905 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.906 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T12:17:15.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T12:17:15.906 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.906 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.906 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.906 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.906 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.906 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.907 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.907 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.907 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.907 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.907 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.907 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T12:17:15.907 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T12:17:15.907 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.907 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.908 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.908 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.908 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.908 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.908 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.908 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.908 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.908 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.908 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.909 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T12:17:15.909 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T12:17:15.909 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.909 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.909 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.909 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.909 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.909 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.910 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.910 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.910 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.910 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.910 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.910 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.910 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T12:17:15.910 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T12:17:15.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.911 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.911 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.911 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.911 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.912 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.912 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T12:17:15.912 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T12:17:15.912 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.912 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.912 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.912 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.913 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.913 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.913 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.913 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.913 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.913 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.914 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.914 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T12:17:15.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T12:17:15.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.914 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.914 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.915 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.915 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.915 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.915 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.915 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.915 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.916 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T12:17:15.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T12:17:15.916 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.916 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.916 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.916 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.916 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.916 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.916 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.917 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.917 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.917 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.917 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.917 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T12:17:15.917 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T12:17:15.917 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.917 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.918 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.918 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.918 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.918 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.918 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.918 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.918 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.919 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.919 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T12:17:15.919 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T12:17:15.919 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.919 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.919 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T12:17:15.920 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.920 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.920 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.920 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.920 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T12:17:15.921 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T12:17:15.921 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.921 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.921 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.921 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.921 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.921 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.922 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.922 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.922 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.922 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-04-25T12:17:15.922 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-04-25T12:17:15.922 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.922 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.923 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.923 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.923 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.923 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.923 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.923 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.923 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.923 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.923 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.924 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.924 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T12:17:15.924 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T12:17:15.924 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.924 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.924 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.924 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T12:17:15.924 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.925 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.925 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.925 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.925 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.925 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.925 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T12:17:15.925 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T12:17:15.926 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.926 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.926 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.926 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.926 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.926 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.926 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.926 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.927 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.927 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.927 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.927 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T12:17:15.927 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T12:17:15.927 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.927 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.927 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.928 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.928 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.928 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.928 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.928 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.928 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.928 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.928 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T12:17:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T12:17:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T12:17:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T12:17:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T12:17:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T12:17:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-04-25T12:17:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T12:17:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T12:17:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-04-25T12:17:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T12:17:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T12:17:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T12:17:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T12:17:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T12:17:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T12:17:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T12:17:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T12:17:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T12:17:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T12:17:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T12:17:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T12:17:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T12:17:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T12:17:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T12:17:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T12:17:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T12:17:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T12:17:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T12:17:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T12:17:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T12:17:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T12:17:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T12:17:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T12:17:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T12:17:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T12:17:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T12:17:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-04-25T12:17:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T12:17:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T12:17:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-04-25T12:17:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T12:17:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T12:17:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T12:17:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T12:17:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-04-25T12:17:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T12:17:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T12:17:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T12:17:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T12:17:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T12:17:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T12:17:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T12:17:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T12:17:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T12:17:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T12:17:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T12:17:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T12:17:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T12:17:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-04-25T12:17:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-04-25T12:17:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-04-25T12:17:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-04-25T12:17:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-04-25T12:17:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-04-25T12:17:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-04-25T12:17:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-04-25T12:17:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-04-25T12:17:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-04-25T12:17:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T12:17:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T12:17:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-04-25T12:17:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-04-25T12:17:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-04-25T12:17:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-04-25T12:17:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T12:17:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T12:17:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T12:17:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T12:17:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T12:17:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T12:17:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-04-25T12:17:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T12:17:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T12:17:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-04-25T12:17:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-04-25T12:17:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-04-25T12:17:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T12:17:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T12:17:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-04-25T12:17:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-04-25T12:17:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T12:17:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T12:17:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T12:17:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T12:17:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T12:17:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-04-25T12:17:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-04-25T12:17:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-25T12:17:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-04-25T12:17:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-04-25T12:17:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-04-25T12:17:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-04-25T12:17:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-04-25T12:17:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-04-25T12:17:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T12:17:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-04-25T12:17:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-04-25T12:17:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-04-25T12:17:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T12:17:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-04-25T12:17:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-04-25T12:17:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-04-25T12:17:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-04-25T12:17:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-04-25T12:17:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-04-25T12:17:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T12:17:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T12:17:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-04-25T12:17:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-04-25T12:17:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.024 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T12:17:16.024 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.024 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.024 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:16.024 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:16.024 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:16.025 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.025 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.025 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:16.025 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.025 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.025 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T12:17:16.025 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.026 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.026 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.026 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-04-25T12:17:16.026 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-04-25T12:17:16.026 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:16.026 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.026 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.027 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-04-25T12:17:16.027 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.027 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.027 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.027 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T12:17:16.027 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.027 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.027 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.028 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.028 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.028 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-04-25T12:17:16.028 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.028 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.028 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.028 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.029 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.029 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.029 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.029 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.029 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.029 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.029 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.030 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.030 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.030 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.030 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.030 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.030 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.030 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.030 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.031 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.031 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.031 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.031 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.031 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.032 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.032 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.032 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.032 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.032 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.032 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.032 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.033 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.033 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.033 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.033 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.033 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.033 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.033 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.033 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.033 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.034 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.034 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.034 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.034 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.034 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.034 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.034 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.034 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.035 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.035 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.035 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.035 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.035 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.035 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.036 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.036 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.036 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.036 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.036 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.036 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.036 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.036 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.037 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-04-25T12:17:16.037 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-04-25T12:17:16.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.037 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.037 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.037 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-04-25T12:17:16.037 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.037 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.038 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.038 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.038 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.038 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-04-25T12:17:16.038 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-04-25T12:17:16.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.039 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.039 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-04-25T12:17:16.039 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.039 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.039 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.039 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.039 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.040 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.040 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.040 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:16.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:16.040 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.040 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.040 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.040 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-04-25T12:17:16.041 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.041 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.041 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.041 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.041 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.041 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.041 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.041 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.042 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.042 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-04-25T12:17:16.042 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-04-25T12:17:16.042 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-04-25T12:17:16.042 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.042 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-04-25T12:17:16.042 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-04-25T12:17:16.043 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.043 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.043 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-04-25T12:17:16.043 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.043 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.043 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T12:17:16.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.044 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.044 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.044 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-04-25T12:17:16.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-04-25T12:17:16.044 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.045 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.045 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.045 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-04-25T12:17:16.045 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.045 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.045 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.045 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T12:17:16.045 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.045 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.046 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.046 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-04-25T12:17:16.046 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-04-25T12:17:16.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.046 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.046 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.046 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.047 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.047 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.047 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.047 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-04-25T12:17:16.047 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.047 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.047 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.047 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:16.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:16.048 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:16.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.048 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-04-25T12:17:16.048 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-04-25T12:17:16.048 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.048 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.048 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T12:17:16.049 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.049 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.049 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.049 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.049 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.049 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.049 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.049 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.050 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.050 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.050 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.050 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.050 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.050 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.050 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.050 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.051 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.051 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.051 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.051 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.051 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.051 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.051 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.052 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.052 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.052 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.052 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.052 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.052 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.052 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.052 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.053 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.053 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.053 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.053 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.053 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.053 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.054 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.054 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.054 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.054 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.054 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.054 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.054 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.055 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.055 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.055 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.055 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.055 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.055 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.055 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.055 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.056 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.056 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.056 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.056 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.056 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.056 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.057 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.057 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.057 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.057 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-04-25T12:17:16.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-04-25T12:17:16.057 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.057 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.057 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.058 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.058 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.058 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.058 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.058 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-04-25T12:17:16.058 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.058 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.059 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.059 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-04-25T12:17:16.059 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-04-25T12:17:16.059 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.059 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.059 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-04-25T12:17:16.059 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.060 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.060 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.060 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-04-25T12:17:16.060 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.060 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.060 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-04-25T12:17:16.060 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-04-25T12:17:16.060 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.061 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.061 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-04-25T12:17:16.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.061 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.061 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T12:17:16.061 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.062 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.062 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.062 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-04-25T12:17:16.062 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-04-25T12:17:16.062 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.062 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.063 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T12:17:16.063 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T12:17:16.063 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-04-25T12:17:16.063 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.063 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T12:17:16.063 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.063 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.064 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-04-25T12:17:16.064 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-04-25T12:17:16.064 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.064 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.064 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.064 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.065 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-04-25T12:17:16.065 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.065 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.065 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-04-25T12:17:16.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-04-25T12:17:16.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.066 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.066 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.066 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-04-25T12:17:16.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.066 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.066 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T12:17:16.066 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.067 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.067 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.067 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.067 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.067 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-04-25T12:17:16.067 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.068 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.068 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.068 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.068 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.068 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.068 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.068 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.068 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.069 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.069 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.069 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.069 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.069 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.069 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.069 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.069 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.069 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.070 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.070 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.070 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.070 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.070 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.070 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.071 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.071 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.071 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.071 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.071 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.071 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.071 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.072 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.072 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.072 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.072 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.072 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.072 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.072 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.072 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.072 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.073 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.073 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.073 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.073 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.073 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.073 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.073 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.073 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.074 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.074 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.074 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.074 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.075 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.075 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.075 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.075 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.075 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.075 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.075 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.075 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.075 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.076 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.076 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.076 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.076 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.076 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-04-25T12:17:16.076 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.076 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.076 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.077 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.077 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.077 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.077 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.077 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.077 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.078 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.078 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.078 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.078 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.078 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.078 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.078 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.078 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.079 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.079 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.079 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.079 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.079 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.079 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.080 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.080 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.080 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.080 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.080 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.080 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.081 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.081 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.081 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.081 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.081 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.082 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.082 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.082 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.082 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.082 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.082 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.082 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.083 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.083 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.083 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.083 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.083 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.084 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.084 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.084 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.084 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.084 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.084 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.085 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.085 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.085 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.085 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.085 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.085 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.086 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-04-25T12:17:16.086 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.086 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.086 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.086 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-04-25T12:17:16.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-04-25T12:17:16.087 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.087 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.087 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.087 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-04-25T12:17:16.087 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.087 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.087 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.087 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T12:17:16.088 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.088 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.088 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-04-25T12:17:16.088 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-04-25T12:17:16.088 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.088 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.088 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:16.089 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-04-25T12:17:16.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.089 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T12:17:16.089 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.089 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.089 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.090 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.090 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.090 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.090 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.090 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.090 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.090 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.091 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.091 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.091 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.091 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.091 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.091 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.091 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.091 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.091 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.092 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.092 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.092 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.092 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.092 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.092 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.092 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.093 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.093 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.093 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.093 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.093 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.093 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.093 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.093 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.094 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.094 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.094 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.094 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.094 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.094 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.095 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.095 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.095 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.095 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.095 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.095 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.095 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.095 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.096 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.096 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.096 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.096 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.096 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.096 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.097 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.097 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.097 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.097 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.097 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.097 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.098 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.098 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.098 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.098 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-04-25T12:17:16.098 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.098 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.098 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.099 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-04-25T12:17:16.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-04-25T12:17:16.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.099 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.099 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-04-25T12:17:16.099 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.099 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.100 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-04-25T12:17:16.100 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T12:17:16.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.100 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.100 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.100 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.101 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.101 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.101 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.101 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.101 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.101 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.101 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.101 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.102 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.102 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.102 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.102 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.102 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.102 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.102 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.102 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.103 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.103 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.103 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.103 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.104 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.104 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.104 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.104 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.104 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.104 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.105 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.105 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.105 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.105 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.105 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.105 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.105 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.105 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.105 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.106 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.106 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.106 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.106 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.106 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.106 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.106 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.106 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.107 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.107 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.107 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.107 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.107 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.107 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.107 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.107 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.108 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.108 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.108 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.108 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.108 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-04-25T12:17:16.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-04-25T12:17:16.108 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.109 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.109 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.109 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.109 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.109 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.109 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-04-25T12:17:16.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T12:17:16.109 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.110 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.110 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-04-25T12:17:16.110 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-04-25T12:17:16.110 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.110 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.110 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.110 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-04-25T12:17:16.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.111 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.111 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-04-25T12:17:16.111 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.111 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.111 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.111 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-04-25T12:17:16.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-04-25T12:17:16.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.112 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.112 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-04-25T12:17:16.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.112 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T12:17:16.112 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.112 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.113 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-04-25T12:17:16.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-04-25T12:17:16.113 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.113 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.113 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.113 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.114 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.114 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.114 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-04-25T12:17:16.114 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.114 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.114 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-04-25T12:17:16.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-04-25T12:17:16.115 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.115 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.115 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-04-25T12:17:16.115 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.115 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.115 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T12:17:16.115 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T12:17:16.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.116 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.116 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.116 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-04-25T12:17:16.116 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.117 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.117 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.117 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.117 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.117 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.117 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.117 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.118 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.118 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.118 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.118 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.118 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.118 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.118 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.118 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.119 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.119 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.119 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.120 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.121 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.121 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.121 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.121 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.121 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.121 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.121 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.122 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.122 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.122 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.122 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.122 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.122 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.122 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.122 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.123 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.123 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.123 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.123 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.123 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.123 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.123 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.124 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.124 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.124 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.124 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.124 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.124 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.124 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.125 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.125 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.125 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.125 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-04-25T12:17:16.125 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.125 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.125 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.125 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.126 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.126 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.126 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.126 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.126 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.126 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.126 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.126 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.127 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.127 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.127 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.127 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.127 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.127 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.127 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.127 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.127 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.128 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.128 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.128 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.128 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.129 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.129 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.129 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.129 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.129 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.130 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.130 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.130 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.130 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.130 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.130 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.130 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.131 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.131 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.131 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.131 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.131 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.132 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.132 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.132 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.132 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.133 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.133 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.133 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.133 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.133 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.133 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-04-25T12:17:16.133 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.134 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.134 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.134 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.134 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.135 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.135 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.135 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.135 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.135 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.135 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.135 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.135 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.135 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.136 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.136 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.136 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.136 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.136 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.136 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.137 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.137 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.137 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.137 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.137 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.137 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.137 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.137 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.138 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.138 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.138 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.138 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.138 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.138 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.139 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.139 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.139 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.139 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.139 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.139 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.139 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.139 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.139 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.140 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.140 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.140 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.140 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.140 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.140 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.140 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.141 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.141 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.141 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.141 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.142 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.142 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.142 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.142 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T12:17:16.142 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.142 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.142 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.143 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-04-25T12:17:16.143 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-04-25T12:17:16.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.143 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.143 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.144 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T12:17:16.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.144 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.144 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.144 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.144 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.145 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.145 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.145 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.145 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.145 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.145 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.145 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.146 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.146 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.146 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.146 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.146 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.146 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.146 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.147 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.147 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.147 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.147 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.147 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.147 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.147 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.148 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.148 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.148 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.148 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.148 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.148 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.148 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.148 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.149 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.149 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.149 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.149 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.149 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.149 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.149 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.150 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.150 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.150 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.150 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.150 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.150 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.150 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.151 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.151 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.151 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.151 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.151 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.152 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.152 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.152 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.152 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.152 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.152 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-04-25T12:17:16.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-04-25T12:17:16.152 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.153 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.153 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.153 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.153 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.153 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-04-25T12:17:16.153 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-04-25T12:17:16.153 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-04-25T12:17:16.154 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.154 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-04-25T12:17:16.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.154 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.154 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.154 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.154 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.155 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.155 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-04-25T12:17:16.155 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.155 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.155 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.155 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.155 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.155 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.156 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.156 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.156 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.156 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.156 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.156 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.157 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.157 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.157 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.157 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.157 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.157 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.157 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.157 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.158 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.158 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.158 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.158 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.158 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.158 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.159 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.159 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.159 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.159 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.159 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.159 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.159 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.160 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.160 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.160 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.160 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.160 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.160 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.160 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.160 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.161 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.161 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.161 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.161 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.161 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.161 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.161 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.162 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.162 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.162 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.162 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.162 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.162 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.162 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.163 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.163 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.163 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.163 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.163 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.163 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.164 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-04-25T12:17:16.164 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.164 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.164 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.164 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.165 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.165 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.165 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.165 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.165 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.166 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.166 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.166 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.166 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.166 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.166 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.166 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.166 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.166 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.167 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.167 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.167 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.167 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.167 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.168 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.168 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.168 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.168 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.168 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.168 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.168 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.169 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.169 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.169 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.169 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.169 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.169 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.169 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.169 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.169 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.169 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.170 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.170 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.170 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.170 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.170 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.170 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.170 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.170 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.171 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.171 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.171 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.171 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.171 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.172 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.172 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.172 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.172 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.172 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.172 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.172 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-04-25T12:17:16.172 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.173 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.173 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.173 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.173 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.173 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.173 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.173 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.173 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.174 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.174 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.174 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.174 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.174 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.174 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.174 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.174 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.174 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.175 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.175 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.175 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.175 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.175 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.175 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.175 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.176 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.176 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.176 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.176 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.176 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.176 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.177 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.177 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.177 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.177 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.177 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.177 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.177 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.177 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.178 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.178 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.178 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.178 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.178 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.178 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.178 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.178 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.178 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.179 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.179 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.179 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.179 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.179 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.179 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.179 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.180 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.180 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.180 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.180 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.180 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.180 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.180 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:16.180 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:16.181 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:16.181 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.181 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:16.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.182 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.182 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.182 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-04-25T12:17:16.182 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-04-25T12:17:16.182 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:16.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-04-25T12:17:16.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-04-25T12:17:16.183 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.183 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-04-25T12:17:16.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T12:17:16.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.183 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.183 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.184 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-04-25T12:17:16.184 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.184 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.184 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.184 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-04-25T12:17:16.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-04-25T12:17:16.184 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.184 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.185 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.185 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.185 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.185 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T12:17:16.185 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.185 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.186 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T12:17:16.186 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T12:17:16.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.186 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:16.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.187 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.187 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.187 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.187 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.188 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.188 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.188 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.188 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.188 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.188 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.188 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.188 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.189 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.189 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.189 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.189 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.190 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.190 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.190 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.191 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.191 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.191 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.191 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.192 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.192 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.192 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.192 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.193 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.193 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.193 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.193 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.193 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.193 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.193 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.193 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.193 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.194 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.194 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.194 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.194 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.194 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.195 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.195 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.195 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.195 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.195 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.195 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-04-25T12:17:16.196 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-04-25T12:17:16.196 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.196 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-04-25T12:17:16.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-04-25T12:17:16.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.197 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-04-25T12:17:16.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-04-25T12:17:16.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:16.197 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.198 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-04-25T12:17:16.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.198 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-04-25T12:17:16.198 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.198 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.199 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.199 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.199 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-04-25T12:17:16.199 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.199 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.199 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.200 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-04-25T12:17:16.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-04-25T12:17:16.200 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.200 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.200 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.200 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:16.200 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.200 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.201 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.201 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.201 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.201 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.201 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-04-25T12:17:16.201 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-04-25T12:17:16.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.201 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.202 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:16.202 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.202 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T12:17:16.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T12:17:16.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.202 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.203 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.203 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.203 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.204 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.204 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.204 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.204 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.204 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.204 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.204 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.204 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.205 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.205 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.205 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.205 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.205 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.206 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.206 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.206 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.206 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.206 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.206 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.206 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.207 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.207 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.207 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.208 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.208 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.208 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.208 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.208 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.208 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.208 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.208 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.209 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.209 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.209 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.209 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.210 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.210 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.210 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.210 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.211 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-04-25T12:17:16.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-04-25T12:17:16.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.211 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.211 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.211 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.211 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.212 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.212 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.212 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.212 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T12:17:16.212 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T12:17:16.212 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.213 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-04-25T12:17:16.213 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.213 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.213 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.213 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.214 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.214 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.214 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-04-25T12:17:16.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-04-25T12:17:16.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:16.214 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.214 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-04-25T12:17:16.215 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.215 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.215 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.215 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.215 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.215 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.215 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.215 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T12:17:16.216 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T12:17:16.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.216 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.216 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-04-25T12:17:16.216 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.216 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T12:17:16.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.217 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.217 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.217 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T12:17:16.217 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T12:17:16.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.217 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.217 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-04-25T12:17:16.218 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.218 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.218 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.218 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T12:17:16.218 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.218 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.218 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.218 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-04-25T12:17:16.219 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-04-25T12:17:16.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.219 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-04-25T12:17:16.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.219 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.220 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.220 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.220 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-04-25T12:17:16.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-04-25T12:17:16.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.220 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-04-25T12:17:16.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.221 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-04-25T12:17:16.221 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-04-25T12:17:16.221 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.221 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.222 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.222 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.222 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.222 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-04-25T12:17:16.222 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-04-25T12:17:16.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.222 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.222 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.223 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-04-25T12:17:16.223 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-04-25T12:17:16.223 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-04-25T12:17:16.223 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.223 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.223 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.224 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-25T12:17:16.224 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.224 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.224 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.224 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.225 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.225 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.225 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.225 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.226 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.226 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.226 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.226 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.226 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.226 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.226 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.227 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.227 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.227 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.227 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.228 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.228 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.228 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.228 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.229 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.229 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.229 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.229 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.229 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.229 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.230 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.230 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.230 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.230 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.230 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.230 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.230 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.230 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.230 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.231 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.231 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.231 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.231 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.231 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.231 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.231 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.231 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.232 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.232 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.232 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-04-25T12:17:16.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-04-25T12:17:16.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.233 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.233 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-04-25T12:17:16.233 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.233 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.233 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.233 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.234 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-04-25T12:17:16.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T12:17:16.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.234 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.234 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.234 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.234 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.234 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.235 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.235 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.235 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-04-25T12:17:16.235 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-04-25T12:17:16.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.236 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.236 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.236 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.236 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.236 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.237 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.237 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.237 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.237 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-04-25T12:17:16.237 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.237 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.237 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-04-25T12:17:16.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-04-25T12:17:16.238 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.238 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.238 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:16.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.239 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.239 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.239 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-04-25T12:17:16.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-04-25T12:17:16.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.239 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.240 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.240 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.240 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.240 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.241 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.241 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.241 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.241 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.241 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.241 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.242 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.242 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.242 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.242 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.242 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.242 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.242 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.243 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.243 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.243 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.243 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.244 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.244 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.244 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.244 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.245 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.245 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.245 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.245 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.245 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.245 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.246 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.246 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.246 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.246 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.246 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.246 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.246 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.246 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.247 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.247 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.247 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.247 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.247 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.247 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.247 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.248 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.248 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-04-25T12:17:16.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-04-25T12:17:16.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.249 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.249 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.250 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-04-25T12:17:16.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-04-25T12:17:16.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.250 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.250 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.250 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.251 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.251 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.251 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.251 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.251 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.252 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-04-25T12:17:16.252 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.252 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.252 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.252 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.252 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.253 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.253 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.253 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.254 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.254 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.254 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.254 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.254 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.254 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.254 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.254 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.255 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.255 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.255 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.255 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.255 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.255 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.255 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.256 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.256 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.256 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.256 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.256 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.257 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.257 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.257 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.257 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.257 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.257 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.257 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.258 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.258 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.258 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.258 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.258 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.259 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.259 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.260 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.260 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.260 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.260 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.260 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.260 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.261 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.261 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-04-25T12:17:16.261 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.261 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.261 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.261 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.261 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.261 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.262 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.262 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.262 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.262 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.262 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.262 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.263 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.263 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.263 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.263 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.263 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.263 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.263 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.264 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.264 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.264 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.264 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.265 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.265 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.265 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.266 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.266 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.266 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.266 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.266 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.267 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.267 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.267 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.267 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.267 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.267 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.267 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.268 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.268 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.268 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.268 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.268 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.269 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.269 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.269 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.269 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.269 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.269 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.269 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.270 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-04-25T12:17:16.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-04-25T12:17:16.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.270 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-04-25T12:17:16.270 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.270 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.270 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.271 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T12:17:16.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.271 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.271 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.271 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.271 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-04-25T12:17:16.272 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.272 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.272 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.272 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.272 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.272 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.272 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.273 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.273 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.273 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.273 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.273 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.273 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.274 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.274 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.274 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.274 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.274 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.274 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.274 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.275 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.275 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.275 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.275 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.275 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.275 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.276 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.276 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.276 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.276 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.277 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.277 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.277 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.277 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.277 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.278 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.278 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.278 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.278 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.278 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.278 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.278 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.279 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.279 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.279 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.279 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.279 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.279 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.280 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.280 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.280 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.280 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-04-25T12:17:16.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-04-25T12:17:16.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.281 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.281 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.281 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.281 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.281 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.281 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.281 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.281 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.281 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.282 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-04-25T12:17:16.282 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-04-25T12:17:16.282 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.282 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.282 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.282 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-04-25T12:17:16.282 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.283 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.283 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.283 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.283 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.283 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-04-25T12:17:16.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-04-25T12:17:16.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.284 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.284 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.284 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.284 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.284 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.284 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.285 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-04-25T12:17:16.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-04-25T12:17:16.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.285 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.285 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.286 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.286 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.286 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.286 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.286 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.286 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.287 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-04-25T12:17:16.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-04-25T12:17:16.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-04-25T12:17:16.287 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.287 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.287 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.287 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.287 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.288 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.288 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.288 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.288 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.288 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.288 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-04-25T12:17:16.288 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-04-25T12:17:16.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.289 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.289 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.289 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.289 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.289 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.289 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.289 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.290 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-04-25T12:17:16.290 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-04-25T12:17:16.290 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.290 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.290 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.290 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.291 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.291 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.291 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.291 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.291 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.291 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.292 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-04-25T12:17:16.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-04-25T12:17:16.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.292 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.292 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.292 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:16.292 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.292 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.292 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.293 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.293 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-04-25T12:17:16.293 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-04-25T12:17:16.293 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.294 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-04-25T12:17:16.294 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-04-25T12:17:16.294 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.294 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.294 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.295 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-04-25T12:17:16.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-04-25T12:17:16.295 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.295 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.295 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.295 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.296 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.296 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.296 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.296 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.296 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.296 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-04-25T12:17:16.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-04-25T12:17:16.297 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.297 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.297 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.297 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.298 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-04-25T12:17:16.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-04-25T12:17:16.298 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.299 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.299 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.300 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.300 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.300 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T12:17:16.300 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.300 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.300 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.301 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-04-25T12:17:16.301 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-04-25T12:17:16.301 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.301 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.301 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.301 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.301 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.301 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.301 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T12:17:16.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.302 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-04-25T12:17:16.302 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-04-25T12:17:16.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.303 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.303 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-04-25T12:17:16.303 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.304 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.304 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.304 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.305 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.305 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.305 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.305 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.305 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.305 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.305 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.305 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.305 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.306 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.307 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.308 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-04-25T12:17:16.312 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.313 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.314 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.316 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.317 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.318 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.321 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.321 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.321 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.321 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.321 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-04-25T12:17:16.322 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.322 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.322 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.322 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.323 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.323 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.323 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.323 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.323 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.323 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.324 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.324 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.324 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.324 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.324 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.324 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.325 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.325 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.325 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.325 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.325 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.325 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.325 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.326 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.326 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.326 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.326 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.326 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.326 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.327 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.327 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.327 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.327 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.327 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.328 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.328 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.328 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.328 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.328 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.329 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.329 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.329 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.329 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.329 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.329 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.330 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.330 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.330 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.330 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.331 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.331 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.331 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.331 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.331 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.331 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.331 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-04-25T12:17:16.332 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.332 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.332 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.332 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-04-25T12:17:16.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-04-25T12:17:16.333 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.333 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.333 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T12:17:16.333 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.333 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.333 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.333 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.334 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.334 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:16.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:16.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:16.334 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.335 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-04-25T12:17:16.335 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.335 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.336 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.336 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.336 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.336 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.336 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.337 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.337 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.337 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.337 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.337 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.337 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.337 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.337 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.338 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.338 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.338 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.338 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.338 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.338 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.338 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.339 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.339 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.339 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.339 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.339 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.339 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.339 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.340 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.340 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.340 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.340 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.340 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.340 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.340 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.341 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.341 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.341 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.341 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.341 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.341 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.341 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.341 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.342 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.342 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.342 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.342 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.343 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.343 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.343 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.343 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.343 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.343 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.344 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-04-25T12:17:16.344 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.344 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.344 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.344 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-04-25T12:17:16.344 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-04-25T12:17:16.344 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.345 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:16.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.345 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.345 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.345 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T12:17:16.345 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.345 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.345 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.346 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-04-25T12:17:16.346 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-04-25T12:17:16.346 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.346 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.346 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:16.346 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.347 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T12:17:16.347 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.347 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-04-25T12:17:16.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-04-25T12:17:16.347 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.348 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.348 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:16.348 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.348 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.348 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T12:17:16.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.349 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.349 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-04-25T12:17:16.349 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-04-25T12:17:16.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.349 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.350 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T12:17:16.350 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.350 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-04-25T12:17:16.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-04-25T12:17:16.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.351 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.351 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.351 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.351 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T12:17:16.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.352 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.352 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-04-25T12:17:16.352 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-04-25T12:17:16.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.352 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.353 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.353 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.353 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.353 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-04-25T12:17:16.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-04-25T12:17:16.353 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.354 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.355 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.355 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.355 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.355 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.355 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.355 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-04-25T12:17:16.355 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-04-25T12:17:16.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.356 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.356 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T12:17:16.356 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-04-25T12:17:16.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-04-25T12:17:16.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.357 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.358 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.358 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.358 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.358 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.358 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.358 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.358 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.359 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:16.359 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:16.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.359 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.359 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.359 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-04-25T12:17:16.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-04-25T12:17:16.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.360 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-04-25T12:17:16.361 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-04-25T12:17:16.361 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.361 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-04-25T12:17:16.361 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.361 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.362 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.362 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.362 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.362 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-04-25T12:17:16.362 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-04-25T12:17:16.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.362 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.363 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.363 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.363 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.364 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-04-25T12:17:16.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-04-25T12:17:16.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.364 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.364 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.365 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.365 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.365 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.365 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.365 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.366 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.366 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.367 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.367 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.367 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.367 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.367 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.367 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.367 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.367 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.368 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.368 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.368 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.368 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.368 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.368 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.368 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.369 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.369 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.369 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.369 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.369 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.369 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.370 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.370 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.370 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.370 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.370 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.370 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.371 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.371 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.371 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.371 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.371 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.371 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.371 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.371 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.372 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.373 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.373 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.373 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.374 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.374 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-04-25T12:17:16.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-04-25T12:17:16.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.375 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.375 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.376 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-04-25T12:17:16.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-04-25T12:17:16.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.376 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.377 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-04-25T12:17:16.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-04-25T12:17:16.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.378 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.378 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.378 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T12:17:16.378 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.378 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.379 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-04-25T12:17:16.379 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.379 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.380 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.380 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.380 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.380 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.381 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.381 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.381 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.381 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.381 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.381 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.381 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.382 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.382 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.382 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.382 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.382 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.383 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.383 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.383 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.383 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.383 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.384 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.384 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.385 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.385 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.385 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.385 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.385 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.385 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.385 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.385 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.386 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.386 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.386 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.387 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.387 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.387 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.387 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.387 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.387 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.388 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.388 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.388 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-25T12:17:16.388 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.389 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.389 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.389 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.389 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.389 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.390 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.390 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.390 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.390 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.390 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.390 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.390 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.391 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.391 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.391 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.391 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.391 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.392 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.392 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.393 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.393 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.393 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.393 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.393 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.394 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.394 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.394 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.394 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.394 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.394 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.395 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.395 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.395 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.395 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.396 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.396 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.396 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.396 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.397 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.397 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-04-25T12:17:16.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-04-25T12:17:16.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-04-25T12:17:16.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.398 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.398 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T12:17:16.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.399 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-04-25T12:17:16.399 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-04-25T12:17:16.399 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.399 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.399 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.400 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.400 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-04-25T12:17:16.400 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.400 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.401 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-04-25T12:17:16.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-04-25T12:17:16.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-04-25T12:17:16.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.402 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T12:17:16.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.402 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-04-25T12:17:16.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T12:17:16.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.403 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-04-25T12:17:16.403 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.403 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.403 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T12:17:16.403 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.404 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.404 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.404 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.404 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-04-25T12:17:16.404 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-04-25T12:17:16.405 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-04-25T12:17:16.405 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-04-25T12:17:16.405 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-04-25T12:17:16.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-04-25T12:17:16.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-04-25T12:17:16.405 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.405 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-04-25T12:17:16.405 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.406 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.406 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.406 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.406 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.406 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.406 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.406 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-04-25T12:17:16.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-04-25T12:17:16.407 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.407 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.407 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.407 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.407 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.407 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.408 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.408 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.408 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-04-25T12:17:16.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-04-25T12:17:16.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-04-25T12:17:16.408 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.409 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-04-25T12:17:16.409 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.409 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.410 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-04-25T12:17:16.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-04-25T12:17:16.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.411 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.411 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.411 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.411 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.411 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.411 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.411 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.412 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.412 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.412 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-04-25T12:17:16.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-04-25T12:17:16.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.413 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.414 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-04-25T12:17:16.414 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-04-25T12:17:16.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.414 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-04-25T12:17:16.414 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.415 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-04-25T12:17:16.415 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-04-25T12:17:16.415 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-04-25T12:17:16.415 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-04-25T12:17:16.415 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-04-25T12:17:16.415 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-04-25T12:17:16.415 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-04-25T12:17:16.415 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.416 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.416 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-04-25T12:17:16.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-04-25T12:17:16.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-04-25T12:17:16.416 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-04-25T12:17:16.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-04-25T12:17:16.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.417 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.417 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.417 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.418 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-04-25T12:17:16.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-04-25T12:17:16.418 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.418 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.418 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.418 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-04-25T12:17:16.418 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.419 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.419 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.419 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-04-25T12:17:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-04-25T12:17:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-04-25T12:17:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-04-25T12:17:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-04-25T12:17:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-04-25T12:17:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-04-25T12:17:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-04-25T12:17:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-04-25T12:17:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T12:17:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-04-25T12:17:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-04-25T12:17:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-04-25T12:17:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-04-25T12:17:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-04-25T12:17:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-04-25T12:17:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-04-25T12:17:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: "services": { 2024-04-25T12:17:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard": "https://172.21.15.19:8443/" 2024-04-25T12:17:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: "always_on_modules": { 2024-04-25T12:17:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus": [ 2024-04-25T12:17:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T12:17:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T12:17:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T12:17:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T12:17:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T12:17:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T12:17:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T12:17:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T12:17:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T12:17:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T12:17:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific": [ 2024-04-25T12:17:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T12:17:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T12:17:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T12:17:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T12:17:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T12:17:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T12:17:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T12:17:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T12:17:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T12:17:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T12:17:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy": [ 2024-04-25T12:17:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T12:17:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T12:17:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T12:17:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T12:17:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T12:17:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T12:17:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T12:17:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T12:17:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T12:17:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T12:17:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "reef": [ 2024-04-25T12:17:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T12:17:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T12:17:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T12:17:16.429 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T12:17:16.429 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T12:17:16.429 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T12:17:16.429 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T12:17:16.429 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T12:17:16.429 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T12:17:16.429 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T12:17:16.429 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-04-25T12:17:16.429 INFO:teuthology.orchestra.run.smithi019.stdout: "squid": [ 2024-04-25T12:17:16.430 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-04-25T12:17:16.430 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-04-25T12:17:16.430 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-04-25T12:17:16.430 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-04-25T12:17:16.430 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-04-25T12:17:16.430 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-04-25T12:17:16.430 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-04-25T12:17:16.430 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-04-25T12:17:16.431 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-04-25T12:17:16.431 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-04-25T12:17:16.431 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T12:17:16.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.431 INFO:teuthology.orchestra.run.smithi019.stdout: "last_failure_osd_epoch": 88, 2024-04-25T12:17:16.431 INFO:teuthology.orchestra.run.smithi019.stdout: "active_clients": [ 2024-04-25T12:17:16.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-04-25T12:17:16.431 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T12:17:16.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T12:17:16.432 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T12:17:16.432 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3932558183 2024-04-25T12:17:16.432 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.432 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T12:17:16.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "libcephsqlite", 2024-04-25T12:17:16.433 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T12:17:16.433 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T12:17:16.433 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T12:17:16.433 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2665942761 2024-04-25T12:17:16.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.433 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T12:17:16.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.434 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-04-25T12:17:16.434 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T12:17:16.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T12:17:16.434 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T12:17:16.434 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 724285221 2024-04-25T12:17:16.435 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.435 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T12:17:16.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T12:17:16.435 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T12:17:16.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T12:17:16.435 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T12:17:16.436 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 742741547 2024-04-25T12:17:16.436 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.436 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T12:17:16.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-04-25T12:17:16.436 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T12:17:16.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.437 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T12:17:16.437 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T12:17:16.437 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2596803504 2024-04-25T12:17:16.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.437 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T12:17:16.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:16.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-04-25T12:17:16.437 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-04-25T12:17:16.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-04-25T12:17:16.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-04-25T12:17:16.438 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-04-25T12:17:16.438 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1541374584 2024-04-25T12:17:16.438 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.438 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T12:17:16.438 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:16.438 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-04-25T12:17:16.439 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:17:16.440 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:14 smithi150 ceph-mon[32250]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T12:17:16.440 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:14 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/475584941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:17:16.440 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:14 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2071755946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:17:16.441 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:14 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3471784275' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T12:17:16.441 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:14 smithi169 ceph-mon[32710]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T12:17:16.441 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:14 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/475584941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:17:16.441 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:14 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2071755946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:17:16.441 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:14 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3471784275' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T12:17:16.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:14.206+0000 7f50892ae640 1 -- 172.21.15.19:0/3471784275 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f506c0a3d20 msgr2=0x7f506c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:16.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:14.206+0000 7f50892ae640 1 --2- 172.21.15.19:0/3471784275 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f506c0a3d20 0x7f506c0a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5084069c90 tx=0x7f507800a4c0 comp rx=0 tx=0).stop 2024-04-25T12:17:16.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:14.206+0000 7f50892ae640 1 -- 172.21.15.19:0/3471784275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50841010a0 msgr2=0x7f5084199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:16.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:14.206+0000 7f50892ae640 1 --2- 172.21.15.19:0/3471784275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50841010a0 0x7f5084199650 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5074006010 tx=0x7f5074004430 comp rx=0 tx=0).stop 2024-04-25T12:17:16.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:14.206+0000 7f50892ae640 1 -- 172.21.15.19:0/3471784275 shutdown_connections 2024-04-25T12:17:16.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:14.206+0000 7f50892ae640 1 --2- 172.21.15.19:0/3471784275 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f506c0a3d20 0x7f506c0a61e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:16.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:14.206+0000 7f50892ae640 1 --2- 172.21.15.19:0/3471784275 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f508419a170 0x7f50841a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:16.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:14.206+0000 7f50892ae640 1 --2- 172.21.15.19:0/3471784275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5084106170 0x7f5084199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:16.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:14.206+0000 7f50892ae640 1 --2- 172.21.15.19:0/3471784275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50841010a0 0x7f5084199650 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:16.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:14.206+0000 7f50892ae640 1 -- 172.21.15.19:0/3471784275 >> 172.21.15.19:0/3471784275 conn(0x7f5084077f50 msgr2=0x7f50840fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:16.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:14.206+0000 7f50892ae640 1 -- 172.21.15.19:0/3471784275 shutdown_connections 2024-04-25T12:17:16.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:14.206+0000 7f50892ae640 1 -- 172.21.15.19:0/3471784275 wait complete. 2024-04-25T12:17:16.453 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T12:17:16.453 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T12:17:16.454 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:14 smithi019 ceph-mon[29213]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T12:17:16.454 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:14 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/475584941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:17:16.454 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:14 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2071755946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:17:16.454 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:14 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3471784275' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T12:17:16.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:16 smithi169 ceph-mon[32710]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-04-25T12:17:16.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:16 smithi169 ceph-mon[32710]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T12:17:16.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:16 smithi150 ceph-mon[32250]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-04-25T12:17:16.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:16 smithi150 ceph-mon[32250]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T12:17:16.714 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:17:16.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:16 smithi019 ceph-mon[29213]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 204 B/s wr, 13 op/s 2024-04-25T12:17:16.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:16 smithi019 ceph-mon[29213]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T12:17:18.609 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:18 smithi019 ceph-mon[29213]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-25T12:17:18.658 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:17:18.658 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T12:17:18.658 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 113, 2024-04-25T12:17:18.658 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 118, 2024-04-25T12:17:18.658 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 44552, 2024-04-25T12:17:18.659 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 94102, 2024-04-25T12:17:18.659 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 15, 2024-04-25T12:17:18.659 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-04-25T12:17:18.659 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.013515987, 2024-04-25T12:17:18.659 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.003537363, 2024-04-25T12:17:18.659 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.008765627, 2024-04-25T12:17:18.659 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000279295, 2024-04-25T12:17:18.659 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:18.660 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 113, 2024-04-25T12:17:18.660 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.014244503, 2024-04-25T12:17:18.660 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000126057 2024-04-25T12:17:18.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.660 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:18.660 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 194, 2024-04-25T12:17:18.660 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000042585, 2024-04-25T12:17:18.660 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000219 2024-04-25T12:17:18.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.661 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T12:17:18.661 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T12:17:18.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.661 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T12:17:18.661 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 285, 2024-04-25T12:17:18.661 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 621, 2024-04-25T12:17:18.661 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 88107, 2024-04-25T12:17:18.662 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 526368, 2024-04-25T12:17:18.662 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 10, 2024-04-25T12:17:18.662 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-04-25T12:17:18.662 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.064239315, 2024-04-25T12:17:18.662 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.030461254, 2024-04-25T12:17:18.662 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.025852074, 2024-04-25T12:17:18.663 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.004665777, 2024-04-25T12:17:18.663 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:18.663 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 621, 2024-04-25T12:17:18.663 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.044526570, 2024-04-25T12:17:18.663 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000071701 2024-04-25T12:17:18.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.663 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:18.663 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 730, 2024-04-25T12:17:18.664 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000226455, 2024-04-25T12:17:18.664 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000310 2024-04-25T12:17:18.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.664 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T12:17:18.664 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 435888 2024-04-25T12:17:18.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.664 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T12:17:18.665 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 329, 2024-04-25T12:17:18.665 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 313, 2024-04-25T12:17:18.665 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1660896, 2024-04-25T12:17:18.665 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 159316, 2024-04-25T12:17:18.665 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-04-25T12:17:18.665 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-04-25T12:17:18.665 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.049590416, 2024-04-25T12:17:18.665 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.016259022, 2024-04-25T12:17:18.666 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.040028945, 2024-04-25T12:17:18.666 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.004727370, 2024-04-25T12:17:18.666 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:18.666 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 313, 2024-04-25T12:17:18.666 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.033108359, 2024-04-25T12:17:18.666 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000105777 2024-04-25T12:17:18.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.666 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:18.667 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 97, 2024-04-25T12:17:18.667 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000022744, 2024-04-25T12:17:18.667 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000234 2024-04-25T12:17:18.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.667 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1636304, 2024-04-25T12:17:18.667 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 131440 2024-04-25T12:17:18.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.667 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T12:17:18.668 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-04-25T12:17:18.668 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T12:17:18.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.668 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-04-25T12:17:18.668 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-25T12:17:18.668 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-25T12:17:18.668 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 41, 2024-04-25T12:17:18.668 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.001000790, 2024-04-25T12:17:18.668 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000024409 2024-04-25T12:17:18.669 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:18.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.669 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-04-25T12:17:18.669 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-25T12:17:18.669 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-25T12:17:18.669 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2, 2024-04-25T12:17:18.669 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000006074, 2024-04-25T12:17:18.669 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000003037 2024-04-25T12:17:18.669 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:18.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.670 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-04-25T12:17:18.670 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-04-25T12:17:18.670 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-04-25T12:17:18.670 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-04-25T12:17:18.670 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.670 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.670 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.671 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-04-25T12:17:18.671 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-04-25T12:17:18.671 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-04-25T12:17:18.671 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-04-25T12:17:18.671 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 11, 2024-04-25T12:17:18.671 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-04-25T12:17:18.671 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-04-25T12:17:18.671 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 13, 2024-04-25T12:17:18.672 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-04-25T12:17:18.672 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 13, 2024-04-25T12:17:18.672 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-04-25T12:17:18.672 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 12, 2024-04-25T12:17:18.672 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-04-25T12:17:18.672 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-04-25T12:17:18.672 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-04-25T12:17:18.672 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 2, 2024-04-25T12:17:18.672 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 2, 2024-04-25T12:17:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 1, 2024-04-25T12:17:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-04-25T12:17:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 1, 2024-04-25T12:17:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-04-25T12:17:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-04-25T12:17:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-04-25T12:17:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-04-25T12:17:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-04-25T12:17:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-04-25T12:17:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-04-25T12:17:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-04-25T12:17:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-04-25T12:17:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-04-25T12:17:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-04-25T12:17:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-04-25T12:17:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 4, 2024-04-25T12:17:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 234, 2024-04-25T12:17:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-04-25T12:17:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T12:17:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-04-25T12:17:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T12:17:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-04-25T12:17:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-04-25T12:17:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T12:17:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T12:17:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-04-25T12:17:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-04-25T12:17:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T12:17:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-04-25T12:17:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T12:17:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T12:17:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T12:17:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-04-25T12:17:18.677 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-04-25T12:17:18.677 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-04-25T12:17:18.677 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-04-25T12:17:18.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.677 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-04-25T12:17:18.677 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-04-25T12:17:18.677 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 3, 2024-04-25T12:17:18.677 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 2, 2024-04-25T12:17:18.677 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 3, 2024-04-25T12:17:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-04-25T12:17:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-04-25T12:17:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-04-25T12:17:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-04-25T12:17:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-04-25T12:17:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-04-25T12:17:18.678 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-04-25T12:17:18.679 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-04-25T12:17:18.679 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-04-25T12:17:18.679 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-04-25T12:17:18.679 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-04-25T12:17:18.679 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-04-25T12:17:18.679 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-04-25T12:17:18.679 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-04-25T12:17:18.679 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-04-25T12:17:18.679 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T12:17:18.680 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-04-25T12:17:18.680 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-04-25T12:17:18.680 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-04-25T12:17:18.680 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-04-25T12:17:18.680 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-04-25T12:17:18.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.680 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-04-25T12:17:18.680 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-04-25T12:17:18.681 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 2, 2024-04-25T12:17:18.681 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-04-25T12:17:18.681 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-04-25T12:17:18.681 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 2, 2024-04-25T12:17:18.681 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-04-25T12:17:18.681 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-04-25T12:17:18.681 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 2, 2024-04-25T12:17:18.681 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-04-25T12:17:18.681 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-04-25T12:17:18.682 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 2, 2024-04-25T12:17:18.682 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-04-25T12:17:18.682 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-04-25T12:17:18.682 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-04-25T12:17:18.682 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-04-25T12:17:18.682 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 4194892, 2024-04-25T12:17:18.682 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 0, 2024-04-25T12:17:18.682 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-04-25T12:17:18.683 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1, 2024-04-25T12:17:18.683 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.001128671, 2024-04-25T12:17:18.683 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001128671 2024-04-25T12:17:18.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.683 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 0 2024-04-25T12:17:18.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.683 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-04-25T12:17:18.683 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 16, 2024-04-25T12:17:18.684 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 16, 2024-04-25T12:17:18.684 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-04-25T12:17:18.684 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 15, 2024-04-25T12:17:18.684 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 16, 2024-04-25T12:17:18.684 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 1, 2024-04-25T12:17:18.684 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 13, 2024-04-25T12:17:18.684 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 13, 2024-04-25T12:17:18.684 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-04-25T12:17:18.684 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-04-25T12:17:18.685 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-04-25T12:17:18.685 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-04-25T12:17:18.685 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 52256, 2024-04-25T12:17:18.685 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 223608 2024-04-25T12:17:18.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.685 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-04-25T12:17:18.685 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-04-25T12:17:18.685 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-04-25T12:17:18.686 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-04-25T12:17:18.686 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-04-25T12:17:18.686 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-04-25T12:17:18.686 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-04-25T12:17:18.686 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.686 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.686 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.687 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-04-25T12:17:18.687 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.687 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.687 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.687 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-04-25T12:17:18.687 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.688 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.688 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.688 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.688 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-04-25T12:17:18.688 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.689 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.689 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.689 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-04-25T12:17:18.689 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.689 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.690 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.690 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-04-25T12:17:18.690 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.690 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.690 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.690 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-04-25T12:17:18.691 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.691 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.691 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.691 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-04-25T12:17:18.691 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.691 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.692 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.692 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-04-25T12:17:18.692 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.692 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.692 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.692 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-04-25T12:17:18.693 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.693 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.693 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.693 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.693 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-04-25T12:17:18.693 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.693 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.693 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.694 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-04-25T12:17:18.694 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.695 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:18 smithi150 ceph-mon[32250]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-25T12:17:18.695 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:18 smithi169 ceph-mon[32710]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-25T12:17:18.695 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.695 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-04-25T12:17:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-04-25T12:17:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.697 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-04-25T12:17:18.697 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.697 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.697 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.697 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-04-25T12:17:18.697 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.697 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.698 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.698 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-04-25T12:17:18.698 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.698 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.698 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.698 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-04-25T12:17:18.699 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.699 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.699 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.699 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-04-25T12:17:18.699 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.699 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.700 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.700 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-04-25T12:17:18.700 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.700 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.700 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.700 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-04-25T12:17:18.701 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.701 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.701 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.701 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-04-25T12:17:18.701 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.701 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.701 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.701 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-04-25T12:17:18.702 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.702 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.702 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.702 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-04-25T12:17:18.702 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.702 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.702 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.703 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-04-25T12:17:18.703 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.703 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.703 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.703 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-04-25T12:17:18.703 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.704 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.704 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.704 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-04-25T12:17:18.704 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.704 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.704 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.704 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-04-25T12:17:18.705 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.705 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.705 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.705 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-04-25T12:17:18.705 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.705 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.705 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.706 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-04-25T12:17:18.706 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-04-25T12:17:18.706 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-04-25T12:17:18.706 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.706 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.706 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.706 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:18.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.707 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-04-25T12:17:18.707 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 0, 2024-04-25T12:17:18.707 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 0, 2024-04-25T12:17:18.707 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 0, 2024-04-25T12:17:18.707 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 0, 2024-04-25T12:17:18.707 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-04-25T12:17:18.707 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-04-25T12:17:18.707 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-04-25T12:17:18.708 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 0, 2024-04-25T12:17:18.708 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-04-25T12:17:18.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.708 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T12:17:18.708 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T12:17:18.708 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T12:17:18.708 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T12:17:18.708 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T12:17:18.709 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T12:17:18.709 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T12:17:18.709 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T12:17:18.709 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T12:17:18.709 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T12:17:18.709 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T12:17:18.709 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T12:17:18.709 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T12:17:18.710 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T12:17:18.710 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T12:17:18.710 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T12:17:18.710 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T12:17:18.710 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T12:17:18.710 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T12:17:18.710 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T12:17:18.710 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T12:17:18.711 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T12:17:18.711 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T12:17:18.711 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T12:17:18.711 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T12:17:18.711 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T12:17:18.711 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T12:17:18.711 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T12:17:18.711 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T12:17:18.712 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T12:17:18.712 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T12:17:18.712 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T12:17:18.712 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T12:17:18.712 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T12:17:18.712 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T12:17:18.712 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T12:17:18.712 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T12:17:18.713 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 238153, 2024-04-25T12:17:18.713 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 77, 2024-04-25T12:17:18.713 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-25T12:17:18.713 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-25T12:17:18.713 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T12:17:18.713 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T12:17:18.713 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T12:17:18.714 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T12:17:18.714 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T12:17:18.714 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T12:17:18.714 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 8344, 2024-04-25T12:17:18.714 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 174, 2024-04-25T12:17:18.714 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T12:17:18.714 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T12:17:18.714 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T12:17:18.715 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T12:17:18.715 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 64808, 2024-04-25T12:17:18.715 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 235, 2024-04-25T12:17:18.715 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T12:17:18.715 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T12:17:18.715 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T12:17:18.715 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T12:17:18.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.715 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-04-25T12:17:18.716 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-04-25T12:17:18.716 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-04-25T12:17:18.716 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 35, 2024-04-25T12:17:18.716 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 10710, 2024-04-25T12:17:18.716 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 33, 2024-04-25T12:17:18.716 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 35, 2024-04-25T12:17:18.716 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-04-25T12:17:18.716 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 35, 2024-04-25T12:17:18.717 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.184998108, 2024-04-25T12:17:18.717 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.005285660 2024-04-25T12:17:18.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.717 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-04-25T12:17:18.717 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-04-25T12:17:18.717 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 35, 2024-04-25T12:17:18.717 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 57 2024-04-25T12:17:18.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.717 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 35, 2024-04-25T12:17:18.718 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 0, 2024-04-25T12:17:18.718 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 35, 2024-04-25T12:17:18.718 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-04-25T12:17:18.718 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-04-25T12:17:18.718 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 0, 2024-04-25T12:17:18.718 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 10, 2024-04-25T12:17:18.718 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 0, 2024-04-25T12:17:18.718 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 1, 2024-04-25T12:17:18.719 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 5, 2024-04-25T12:17:18.719 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-04-25T12:17:18.719 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-04-25T12:17:18.719 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-04-25T12:17:18.719 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-04-25T12:17:18.719 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 6, 2024-04-25T12:17:18.719 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-04-25T12:17:18.719 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-04-25T12:17:18.719 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-04-25T12:17:18.720 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 0, 2024-04-25T12:17:18.720 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 20, 2024-04-25T12:17:18.720 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-04-25T12:17:18.720 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-04-25T12:17:18.720 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-04-25T12:17:18.720 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-04-25T12:17:18.720 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-04-25T12:17:18.720 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-04-25T12:17:18.720 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T12:17:18.721 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-04-25T12:17:18.721 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-04-25T12:17:18.721 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-04-25T12:17:18.721 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-04-25T12:17:18.721 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-04-25T12:17:18.721 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-04-25T12:17:18.721 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-04-25T12:17:18.721 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-04-25T12:17:18.721 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-04-25T12:17:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-04-25T12:17:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-04-25T12:17:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-04-25T12:17:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-04-25T12:17:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-04-25T12:17:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-04-25T12:17:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-04-25T12:17:18.722 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-04-25T12:17:18.723 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-04-25T12:17:18.723 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-04-25T12:17:18.723 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 96, 2024-04-25T12:17:18.723 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-04-25T12:17:18.723 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 11, 2024-04-25T12:17:18.723 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 9, 2024-04-25T12:17:18.723 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 9, 2024-04-25T12:17:18.723 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-04-25T12:17:18.723 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-04-25T12:17:18.724 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 14, 2024-04-25T12:17:18.724 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 0, 2024-04-25T12:17:18.724 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 1 2024-04-25T12:17:18.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.724 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-04-25T12:17:18.724 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 1, 2024-04-25T12:17:18.724 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-04-25T12:17:18.724 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-04-25T12:17:18.725 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-04-25T12:17:18.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.725 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-04-25T12:17:18.725 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-04-25T12:17:18.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.725 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-04-25T12:17:18.725 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-04-25T12:17:18.726 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T12:17:18.726 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-04-25T12:17:18.726 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-04-25T12:17:18.726 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-04-25T12:17:18.726 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-04-25T12:17:18.726 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-04-25T12:17:18.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.726 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T12:17:18.727 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T12:17:18.727 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T12:17:18.727 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T12:17:18.727 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 723, 2024-04-25T12:17:18.727 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1712303, 2024-04-25T12:17:18.727 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:18.727 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 723, 2024-04-25T12:17:18.727 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T12:17:18.727 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T12:17:18.728 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 723, 2024-04-25T12:17:18.728 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1712303, 2024-04-25T12:17:18.728 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T12:17:18.728 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.728 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.728 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.728 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:18.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.729 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-04-25T12:17:18.729 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T12:17:18.729 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T12:17:18.729 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T12:17:18.729 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T12:17:18.729 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T12:17:18.729 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:18.729 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:18.729 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 35, 2024-04-25T12:17:18.730 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 10710, 2024-04-25T12:17:18.730 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 35, 2024-04-25T12:17:18.730 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 10710, 2024-04-25T12:17:18.730 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T12:17:18.730 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.730 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.730 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.730 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:18.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.731 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-04-25T12:17:18.731 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T12:17:18.731 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-04-25T12:17:18.731 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T12:17:18.731 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T12:17:18.731 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T12:17:18.731 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:18.731 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:18.732 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 35, 2024-04-25T12:17:18.732 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 35, 2024-04-25T12:17:18.732 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 35, 2024-04-25T12:17:18.732 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 35, 2024-04-25T12:17:18.732 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T12:17:18.732 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.732 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.732 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.733 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:18.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.733 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-04-25T12:17:18.733 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T12:17:18.733 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-25T12:17:18.733 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T12:17:18.733 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T12:17:18.734 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T12:17:18.734 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:18.734 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:18.734 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T12:17:18.734 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T12:17:18.734 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-04-25T12:17:18.734 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-04-25T12:17:18.734 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T12:17:18.735 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.735 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.735 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.735 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:18.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:18.735 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x562f9dcaa500": { 2024-04-25T12:17:18.735 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T12:17:18.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-25T12:17:18.736 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T12:17:18.736 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 2, 2024-04-25T12:17:18.736 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 588, 2024-04-25T12:17:18.736 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:18.736 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 2, 2024-04-25T12:17:18.736 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T12:17:18.736 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T12:17:18.736 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 1, 2024-04-25T12:17:18.736 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 588, 2024-04-25T12:17:18.737 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T12:17:18.737 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:18.737 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:18.737 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:18.737 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:18.737 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:18.737 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:17:19.497 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-25T12:17:19.498 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-25T12:17:19.498 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-25T12:17:19.498 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=3 2024-04-25T12:17:19.498 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-25T12:17:19.498 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-25T12:17:19.498 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T12:17:19.499 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T12:17:19.499 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T12:17:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:17:20.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:20 smithi169 ceph-mon[32710]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 208 B/s wr, 13 op/s 2024-04-25T12:17:20.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:20 smithi150 ceph-mon[32250]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 208 B/s wr, 13 op/s 2024-04-25T12:17:20.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:20 smithi019 ceph-mon[29213]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 208 B/s wr, 13 op/s 2024-04-25T12:17:21.550 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:17:21.551 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T12:17:21.551 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 217, 2024-04-25T12:17:21.551 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 206, 2024-04-25T12:17:21.551 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1588336, 2024-04-25T12:17:21.551 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 126624, 2024-04-25T12:17:21.551 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-04-25T12:17:21.551 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-04-25T12:17:21.551 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.046245683, 2024-04-25T12:17:21.551 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.014761606, 2024-04-25T12:17:21.552 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.034425000, 2024-04-25T12:17:21.552 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.004614476, 2024-04-25T12:17:21.552 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:21.552 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 205, 2024-04-25T12:17:21.552 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.011454639, 2024-04-25T12:17:21.552 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000055876 2024-04-25T12:17:21.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:21.552 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:21.552 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:21.553 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:21.553 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:21.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:21.553 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1588336, 2024-04-25T12:17:21.553 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 126624 2024-04-25T12:17:21.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:21.553 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T12:17:21.553 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 6, 2024-04-25T12:17:21.554 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 160, 2024-04-25T12:17:21.554 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 3312, 2024-04-25T12:17:21.554 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 184336, 2024-04-25T12:17:21.554 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-04-25T12:17:21.554 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-25T12:17:21.554 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.015411910, 2024-04-25T12:17:21.554 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.010638569, 2024-04-25T12:17:21.554 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002290607, 2024-04-25T12:17:21.555 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T12:17:21.555 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:21.555 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 158, 2024-04-25T12:17:21.555 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.016699515, 2024-04-25T12:17:21.555 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000105693 2024-04-25T12:17:21.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:21.555 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:21.555 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:21.555 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:21.556 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:21.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:21.556 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-25T12:17:21.556 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 184336 2024-04-25T12:17:21.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:21.556 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T12:17:21.556 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-04-25T12:17:21.556 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-04-25T12:17:21.556 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-04-25T12:17:21.557 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-04-25T12:17:21.557 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-04-25T12:17:21.557 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-04-25T12:17:21.557 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.000419829, 2024-04-25T12:17:21.557 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T12:17:21.557 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000331343, 2024-04-25T12:17:21.557 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T12:17:21.557 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:21.557 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:21.558 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:21.558 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:21.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:21.558 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:21.558 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:21.558 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:21.558 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:21.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:21.559 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T12:17:21.559 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T12:17:21.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:21.559 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T12:17:21.559 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-04-25T12:17:21.559 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T12:17:21.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:21.559 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T12:17:21.559 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T12:17:21.560 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T12:17:21.560 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T12:17:21.560 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T12:17:21.560 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T12:17:21.560 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T12:17:21.560 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T12:17:21.560 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T12:17:21.560 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T12:17:21.560 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T12:17:21.561 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T12:17:21.561 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T12:17:21.561 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T12:17:21.561 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T12:17:21.561 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T12:17:21.561 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T12:17:21.561 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T12:17:21.561 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T12:17:21.562 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T12:17:21.562 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T12:17:21.562 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T12:17:21.562 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T12:17:21.562 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T12:17:21.562 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T12:17:21.562 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T12:17:21.562 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T12:17:21.563 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T12:17:21.563 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T12:17:21.563 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T12:17:21.563 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T12:17:21.563 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T12:17:21.563 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T12:17:21.563 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T12:17:21.563 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T12:17:21.563 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T12:17:21.564 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T12:17:21.564 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 148994, 2024-04-25T12:17:21.564 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 31, 2024-04-25T12:17:21.564 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-25T12:17:21.564 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-25T12:17:21.564 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T12:17:21.564 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T12:17:21.564 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T12:17:21.564 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T12:17:21.565 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T12:17:21.565 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T12:17:21.565 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-04-25T12:17:21.565 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-04-25T12:17:21.565 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T12:17:21.565 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T12:17:21.565 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T12:17:21.565 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T12:17:21.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-04-25T12:17:21.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-04-25T12:17:21.566 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T12:17:21.566 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T12:17:21.566 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T12:17:21.567 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T12:17:21.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:21.567 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T12:17:21.567 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T12:17:21.567 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T12:17:21.567 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T12:17:21.567 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 219, 2024-04-25T12:17:21.567 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1557753, 2024-04-25T12:17:21.567 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:21.568 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 219, 2024-04-25T12:17:21.568 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T12:17:21.568 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T12:17:21.568 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 219, 2024-04-25T12:17:21.568 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1557753, 2024-04-25T12:17:21.568 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T12:17:21.568 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:21.568 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:21.569 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:21.569 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:21.569 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:21.569 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:17:22.245 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T12:17:22.245 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T12:17:22.245 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T12:17:22.245 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T12:17:22.245 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T12:17:22.245 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T12:17:22.245 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T12:17:22.246 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T12:17:22.246 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T12:17:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:17:22.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:22 smithi019 ceph-mon[29213]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-25T12:17:22.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:22 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:17:22.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:22 smithi169 ceph-mon[32710]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-25T12:17:22.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:22 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:17:22.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:22 smithi150 ceph-mon[32250]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-25T12:17:22.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:22 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T12:17:24.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:24 smithi169 ceph-mon[32710]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-25T12:17:24.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:24 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:24.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:24 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:24.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:24 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:24.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:24 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:24.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:24 smithi150 ceph-mon[32250]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-25T12:17:24.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:24 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:24.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:24 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:24.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:24 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:24.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:24 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:24.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:24 smithi019 ceph-mon[29213]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-25T12:17:24.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:24 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:24.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:24 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:24.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:24 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:24.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:24 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:24.720 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:17:24.720 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T12:17:24.720 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 738, 2024-04-25T12:17:24.720 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 737, 2024-04-25T12:17:24.720 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 249651, 2024-04-25T12:17:24.720 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 233543, 2024-04-25T12:17:24.721 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-04-25T12:17:24.721 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-04-25T12:17:24.721 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.089083732, 2024-04-25T12:17:24.721 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.037137981, 2024-04-25T12:17:24.721 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.018914102, 2024-04-25T12:17:24.721 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.022647472, 2024-04-25T12:17:24.721 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:24.721 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 736, 2024-04-25T12:17:24.722 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.044440603, 2024-04-25T12:17:24.722 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000060381 2024-04-25T12:17:24.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.722 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:24.722 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.722 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.722 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.723 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T12:17:24.723 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T12:17:24.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.723 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T12:17:24.723 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 965, 2024-04-25T12:17:24.723 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 953, 2024-04-25T12:17:24.723 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1809696, 2024-04-25T12:17:24.723 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 360941, 2024-04-25T12:17:24.723 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-04-25T12:17:24.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-25T12:17:24.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.106425850, 2024-04-25T12:17:24.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.033355346, 2024-04-25T12:17:24.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.054087769, 2024-04-25T12:17:24.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.023547015, 2024-04-25T12:17:24.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:24.724 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 952, 2024-04-25T12:17:24.724 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.033357392, 2024-04-25T12:17:24.725 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000035039 2024-04-25T12:17:24.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.725 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:24.725 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.725 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.725 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.725 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1616768, 2024-04-25T12:17:24.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 127472 2024-04-25T12:17:24.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.726 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T12:17:24.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 5, 2024-04-25T12:17:24.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 157, 2024-04-25T12:17:24.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1668, 2024-04-25T12:17:24.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 437029, 2024-04-25T12:17:24.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-04-25T12:17:24.727 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-04-25T12:17:24.727 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.012720640, 2024-04-25T12:17:24.727 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.008244518, 2024-04-25T12:17:24.727 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002313786, 2024-04-25T12:17:24.727 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000055777, 2024-04-25T12:17:24.727 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:24.727 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 154, 2024-04-25T12:17:24.727 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.011769739, 2024-04-25T12:17:24.727 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000076426 2024-04-25T12:17:24.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.728 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:24.728 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.728 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.728 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.728 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T12:17:24.728 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 436064 2024-04-25T12:17:24.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.729 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T12:17:24.729 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-04-25T12:17:24.729 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T12:17:24.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.729 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-04-25T12:17:24.729 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-25T12:17:24.729 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-25T12:17:24.729 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2933, 2024-04-25T12:17:24.730 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.079750384, 2024-04-25T12:17:24.730 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000027190 2024-04-25T12:17:24.730 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:24.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.730 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-04-25T12:17:24.730 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-04-25T12:17:24.730 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-04-25T12:17:24.730 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.731 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.731 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.731 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:24.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.731 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-04-25T12:17:24.731 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-04-25T12:17:24.731 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-04-25T12:17:24.732 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-04-25T12:17:24.732 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.732 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.732 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.732 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-04-25T12:17:24.732 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-04-25T12:17:24.732 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-04-25T12:17:24.732 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-04-25T12:17:24.733 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-04-25T12:17:24.733 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-04-25T12:17:24.733 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-04-25T12:17:24.733 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 0, 2024-04-25T12:17:24.733 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-04-25T12:17:24.733 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 0, 2024-04-25T12:17:24.733 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-04-25T12:17:24.733 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 0, 2024-04-25T12:17:24.734 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-04-25T12:17:24.734 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-04-25T12:17:24.734 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-04-25T12:17:24.734 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 1, 2024-04-25T12:17:24.734 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-04-25T12:17:24.734 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-04-25T12:17:24.734 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-04-25T12:17:24.734 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-04-25T12:17:24.735 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-04-25T12:17:24.735 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-04-25T12:17:24.735 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-04-25T12:17:24.735 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-04-25T12:17:24.735 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-04-25T12:17:24.735 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-04-25T12:17:24.735 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-04-25T12:17:24.735 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-04-25T12:17:24.735 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-04-25T12:17:24.736 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-04-25T12:17:24.736 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-04-25T12:17:24.736 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-04-25T12:17:24.736 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-04-25T12:17:24.736 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-04-25T12:17:24.736 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-04-25T12:17:24.736 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T12:17:24.736 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-04-25T12:17:24.736 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T12:17:24.737 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-04-25T12:17:24.737 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-04-25T12:17:24.737 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T12:17:24.737 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T12:17:24.737 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-04-25T12:17:24.737 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-04-25T12:17:24.737 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T12:17:24.737 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-04-25T12:17:24.737 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T12:17:24.738 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T12:17:24.738 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T12:17:24.738 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-04-25T12:17:24.738 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-04-25T12:17:24.738 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-04-25T12:17:24.738 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-04-25T12:17:24.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.738 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-04-25T12:17:24.739 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-04-25T12:17:24.739 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-04-25T12:17:24.739 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-04-25T12:17:24.739 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-04-25T12:17:24.739 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-04-25T12:17:24.739 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-04-25T12:17:24.739 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-04-25T12:17:24.739 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-04-25T12:17:24.740 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-04-25T12:17:24.740 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-04-25T12:17:24.740 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-04-25T12:17:24.740 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-04-25T12:17:24.740 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-04-25T12:17:24.740 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-04-25T12:17:24.740 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-04-25T12:17:24.740 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-04-25T12:17:24.740 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-04-25T12:17:24.741 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-04-25T12:17:24.741 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-04-25T12:17:24.741 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T12:17:24.741 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-04-25T12:17:24.741 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-04-25T12:17:24.741 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-04-25T12:17:24.741 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-04-25T12:17:24.741 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-04-25T12:17:24.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.742 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-04-25T12:17:24.742 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-04-25T12:17:24.742 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-04-25T12:17:24.742 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-04-25T12:17:24.742 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-04-25T12:17:24.742 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 2, 2024-04-25T12:17:24.742 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-04-25T12:17:24.743 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-04-25T12:17:24.743 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-04-25T12:17:24.743 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-04-25T12:17:24.743 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-04-25T12:17:24.743 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 2, 2024-04-25T12:17:24.743 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-04-25T12:17:24.743 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-04-25T12:17:24.743 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-04-25T12:17:24.743 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-04-25T12:17:24.744 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 4194892, 2024-04-25T12:17:24.744 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 4194352, 2024-04-25T12:17:24.744 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-04-25T12:17:24.744 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.744 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.744 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.744 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 2 2024-04-25T12:17:24.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.745 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-04-25T12:17:24.745 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 3, 2024-04-25T12:17:24.745 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 3, 2024-04-25T12:17:24.745 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 0, 2024-04-25T12:17:24.745 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 1, 2024-04-25T12:17:24.745 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 1, 2024-04-25T12:17:24.745 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-04-25T12:17:24.745 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 0, 2024-04-25T12:17:24.746 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 0, 2024-04-25T12:17:24.746 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-04-25T12:17:24.746 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-04-25T12:17:24.746 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-04-25T12:17:24.746 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-04-25T12:17:24.746 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 49992, 2024-04-25T12:17:24.746 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215416 2024-04-25T12:17:24.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.746 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-04-25T12:17:24.747 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-04-25T12:17:24.747 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-04-25T12:17:24.747 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-04-25T12:17:24.747 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-04-25T12:17:24.747 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-04-25T12:17:24.747 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-04-25T12:17:24.747 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.747 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.747 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.748 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-04-25T12:17:24.748 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.748 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.748 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.748 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-04-25T12:17:24.748 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.749 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.749 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.749 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-04-25T12:17:24.749 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.749 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.749 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.749 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-04-25T12:17:24.750 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.750 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.750 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.750 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-04-25T12:17:24.750 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.750 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.750 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.751 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-04-25T12:17:24.751 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.751 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.751 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.751 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-04-25T12:17:24.751 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.752 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.752 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.752 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-04-25T12:17:24.752 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.752 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.752 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.752 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-04-25T12:17:24.753 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.753 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.753 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.753 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-04-25T12:17:24.753 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.753 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.753 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.754 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-04-25T12:17:24.754 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.754 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.754 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.755 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-04-25T12:17:24.755 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.755 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.755 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.755 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-04-25T12:17:24.755 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.756 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.756 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.756 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-04-25T12:17:24.756 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.756 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.756 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.757 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-04-25T12:17:24.757 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.757 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.757 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.757 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-04-25T12:17:24.757 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.757 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.757 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.758 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-04-25T12:17:24.758 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.758 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.758 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.758 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-04-25T12:17:24.758 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.759 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.759 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.759 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-04-25T12:17:24.759 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.759 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.759 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.760 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-04-25T12:17:24.760 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.760 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.760 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.760 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-04-25T12:17:24.760 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.760 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.760 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.761 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-04-25T12:17:24.761 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.761 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.761 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.761 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-04-25T12:17:24.761 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.762 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.762 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.762 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-04-25T12:17:24.762 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.762 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.762 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-04-25T12:17:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-04-25T12:17:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.763 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.764 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-04-25T12:17:24.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.764 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.764 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-04-25T12:17:24.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.765 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.765 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-04-25T12:17:24.765 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-04-25T12:17:24.765 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-04-25T12:17:24.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.765 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-04-25T12:17:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 0, 2024-04-25T12:17:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 0, 2024-04-25T12:17:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 0, 2024-04-25T12:17:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 0, 2024-04-25T12:17:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-04-25T12:17:24.767 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-04-25T12:17:24.767 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-04-25T12:17:24.767 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 0, 2024-04-25T12:17:24.767 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-04-25T12:17:24.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.767 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T12:17:24.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T12:17:24.767 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T12:17:24.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T12:17:24.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T12:17:24.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T12:17:24.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T12:17:24.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T12:17:24.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T12:17:24.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T12:17:24.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T12:17:24.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T12:17:24.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T12:17:24.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T12:17:24.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T12:17:24.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T12:17:24.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T12:17:24.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T12:17:24.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T12:17:24.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T12:17:24.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T12:17:24.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T12:17:24.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T12:17:24.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T12:17:24.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T12:17:24.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T12:17:24.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T12:17:24.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T12:17:24.771 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T12:17:24.771 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T12:17:24.771 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T12:17:24.771 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T12:17:24.771 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T12:17:24.771 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T12:17:24.771 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T12:17:24.771 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T12:17:24.771 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T12:17:24.772 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 197473, 2024-04-25T12:17:24.772 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 55, 2024-04-25T12:17:24.772 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-25T12:17:24.772 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-25T12:17:24.772 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T12:17:24.772 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T12:17:24.772 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T12:17:24.772 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T12:17:24.773 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T12:17:24.773 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T12:17:24.773 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 8224, 2024-04-25T12:17:24.773 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 171, 2024-04-25T12:17:24.773 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T12:17:24.773 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T12:17:24.773 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T12:17:24.773 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T12:17:24.773 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 8512, 2024-04-25T12:17:24.774 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 13, 2024-04-25T12:17:24.774 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T12:17:24.774 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T12:17:24.774 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T12:17:24.774 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T12:17:24.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.774 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-04-25T12:17:24.774 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-04-25T12:17:24.775 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-04-25T12:17:24.775 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 1470, 2024-04-25T12:17:24.775 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-04-25T12:17:24.775 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-04-25T12:17:24.775 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 1470, 2024-04-25T12:17:24.775 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-04-25T12:17:24.775 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1470, 2024-04-25T12:17:24.775 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.479984615, 2024-04-25T12:17:24.775 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001006792 2024-04-25T12:17:24.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.776 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-04-25T12:17:24.776 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-04-25T12:17:24.776 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1470, 2024-04-25T12:17:24.776 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1471 2024-04-25T12:17:24.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.776 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 1470, 2024-04-25T12:17:24.776 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 1470, 2024-04-25T12:17:24.777 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-04-25T12:17:24.777 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-04-25T12:17:24.777 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-04-25T12:17:24.777 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 732, 2024-04-25T12:17:24.777 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-04-25T12:17:24.777 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 736, 2024-04-25T12:17:24.777 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-04-25T12:17:24.777 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-04-25T12:17:24.777 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-04-25T12:17:24.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-04-25T12:17:24.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-04-25T12:17:24.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-04-25T12:17:24.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-04-25T12:17:24.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-04-25T12:17:24.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-04-25T12:17:24.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-04-25T12:17:24.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-04-25T12:17:24.779 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-04-25T12:17:24.779 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-04-25T12:17:24.779 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-04-25T12:17:24.779 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-04-25T12:17:24.779 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-04-25T12:17:24.779 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-04-25T12:17:24.779 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-04-25T12:17:24.779 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T12:17:24.779 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-04-25T12:17:24.780 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-04-25T12:17:24.780 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-04-25T12:17:24.780 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-04-25T12:17:24.780 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-04-25T12:17:24.780 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-04-25T12:17:24.780 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-04-25T12:17:24.780 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-04-25T12:17:24.780 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-04-25T12:17:24.781 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-04-25T12:17:24.781 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-04-25T12:17:24.781 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-04-25T12:17:24.781 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-04-25T12:17:24.781 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-04-25T12:17:24.781 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-04-25T12:17:24.781 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-04-25T12:17:24.781 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-04-25T12:17:24.782 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-04-25T12:17:24.782 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-04-25T12:17:24.782 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 96, 2024-04-25T12:17:24.782 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-04-25T12:17:24.782 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 11, 2024-04-25T12:17:24.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 6, 2024-04-25T12:17:24.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 6, 2024-04-25T12:17:24.782 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-04-25T12:17:24.783 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-04-25T12:17:24.783 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-04-25T12:17:24.783 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-04-25T12:17:24.783 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-04-25T12:17:24.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.783 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-04-25T12:17:24.783 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-04-25T12:17:24.783 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-04-25T12:17:24.783 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-04-25T12:17:24.784 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-04-25T12:17:24.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.784 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-04-25T12:17:24.784 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-04-25T12:17:24.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.784 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-04-25T12:17:24.784 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-04-25T12:17:24.784 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T12:17:24.784 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-04-25T12:17:24.785 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-04-25T12:17:24.785 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-04-25T12:17:24.785 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-04-25T12:17:24.785 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-04-25T12:17:24.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.785 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T12:17:24.785 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T12:17:24.785 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T12:17:24.786 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T12:17:24.786 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 1704, 2024-04-25T12:17:24.786 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1892833, 2024-04-25T12:17:24.786 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:24.786 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 1704, 2024-04-25T12:17:24.786 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T12:17:24.786 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T12:17:24.786 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 1704, 2024-04-25T12:17:24.786 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1892833, 2024-04-25T12:17:24.787 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T12:17:24.787 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.787 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.787 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.787 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:24.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.787 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-04-25T12:17:24.787 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T12:17:24.788 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T12:17:24.788 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T12:17:24.788 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T12:17:24.788 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T12:17:24.788 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:24.788 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:24.788 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 1470, 2024-04-25T12:17:24.788 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 593, 2024-04-25T12:17:24.788 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 1470, 2024-04-25T12:17:24.789 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 593, 2024-04-25T12:17:24.789 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T12:17:24.789 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.789 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.789 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.789 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:24.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.789 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-04-25T12:17:24.790 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T12:17:24.790 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-04-25T12:17:24.790 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T12:17:24.790 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T12:17:24.790 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T12:17:24.790 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:24.790 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:24.790 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 1470, 2024-04-25T12:17:24.791 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 1470, 2024-04-25T12:17:24.791 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 1470, 2024-04-25T12:17:24.791 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1470, 2024-04-25T12:17:24.791 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T12:17:24.791 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.791 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.791 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.791 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:24.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.792 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-04-25T12:17:24.792 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T12:17:24.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-25T12:17:24.792 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T12:17:24.792 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T12:17:24.792 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T12:17:24.792 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:24.792 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:24.792 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T12:17:24.793 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T12:17:24.793 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-04-25T12:17:24.793 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-04-25T12:17:24.793 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T12:17:24.793 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.793 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.793 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.793 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:24.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:24.794 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55dcd30606e0": { 2024-04-25T12:17:24.794 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T12:17:24.794 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-04-25T12:17:24.794 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T12:17:24.794 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-04-25T12:17:24.794 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-04-25T12:17:24.794 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:24.794 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:24.795 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T12:17:24.795 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T12:17:24.795 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-04-25T12:17:24.795 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-04-25T12:17:24.795 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T12:17:24.795 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:24.795 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:24.795 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:24.795 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:24.796 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:24.796 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:17:25.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:25 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:25 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:25 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:25 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:25 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-25T12:17:25.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:25 smithi169 ceph-mon[32710]: Unable to set osd_memory_target on smithi150 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T12:17:25.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:25 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:25 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:25 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:25 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:25 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T12:17:25.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:25 smithi169 ceph-mon[32710]: Unable to set osd_memory_target on smithi169 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T12:17:25.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:25 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:25 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:25 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:25 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:25 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-25T12:17:25.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:25 smithi150 ceph-mon[32250]: Unable to set osd_memory_target on smithi150 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T12:17:25.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:25 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:25 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:25 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:25 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:25 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T12:17:25.668 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:25 smithi150 ceph-mon[32250]: Unable to set osd_memory_target on smithi169 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T12:17:25.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:25 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:25 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:25 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:25 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:25 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi150 to 176.2M 2024-04-25T12:17:25.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:25 smithi019 ceph-mon[29213]: Unable to set osd_memory_target on smithi150 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T12:17:25.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:25 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:25 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:25 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:25 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:25.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:25 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi169 to 176.2M 2024-04-25T12:17:25.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:25 smithi019 ceph-mon[29213]: Unable to set osd_memory_target on smithi169 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T12:17:25.716 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-25T12:17:25.717 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-25T12:17:25.717 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-25T12:17:25.717 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-04-25T12:17:25.717 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-25T12:17:25.717 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-25T12:17:25.717 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T12:17:25.717 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T12:17:25.718 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T12:17:25.994 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T12:17:26.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:26 smithi169 ceph-mon[32710]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:17:26.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:26 smithi150 ceph-mon[32250]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:17:26.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:26 smithi019 ceph-mon[29213]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T12:17:27.692 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-04-25T12:17:27.693 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T12:17:27.693 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-04-25T12:17:27.693 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-04-25T12:17:27.693 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-04-25T12:17:27.693 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-04-25T12:17:27.693 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-04-25T12:17:27.694 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-04-25T12:17:27.694 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.001416908, 2024-04-25T12:17:27.694 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T12:17:27.694 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000278968, 2024-04-25T12:17:27.694 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T12:17:27.694 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:27.694 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:27.694 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:27.695 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:27.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:27.695 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:27.695 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:27.695 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:27.695 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:27.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:27.695 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T12:17:27.696 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T12:17:27.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:27.696 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T12:17:27.696 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 224, 2024-04-25T12:17:27.696 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 211, 2024-04-25T12:17:27.696 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1592144, 2024-04-25T12:17:27.696 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 128992, 2024-04-25T12:17:27.696 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-04-25T12:17:27.696 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-04-25T12:17:27.696 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.045575442, 2024-04-25T12:17:27.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.015063555, 2024-04-25T12:17:27.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.034998469, 2024-04-25T12:17:27.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.004647429, 2024-04-25T12:17:27.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:27.697 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 211, 2024-04-25T12:17:27.697 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.012878281, 2024-04-25T12:17:27.697 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000061034 2024-04-25T12:17:27.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:27.697 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:27.698 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:27.698 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:27.698 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:27.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:27.698 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1592144, 2024-04-25T12:17:27.698 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 128992 2024-04-25T12:17:27.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:27.699 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T12:17:27.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2, 2024-04-25T12:17:27.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 160, 2024-04-25T12:17:27.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 320, 2024-04-25T12:17:27.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 184240, 2024-04-25T12:17:27.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-04-25T12:17:27.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-04-25T12:17:27.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.015020551, 2024-04-25T12:17:27.699 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.009977877, 2024-04-25T12:17:27.700 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002418022, 2024-04-25T12:17:27.700 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T12:17:27.700 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:27.700 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 157, 2024-04-25T12:17:27.700 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.014470800, 2024-04-25T12:17:27.700 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000092170 2024-04-25T12:17:27.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:27.700 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:27.701 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:27.701 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:27.701 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:27.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:27.701 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T12:17:27.701 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 184240 2024-04-25T12:17:27.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:27.701 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-04-25T12:17:27.701 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-04-25T12:17:27.702 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-04-25T12:17:27.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:27.702 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-04-25T12:17:27.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-04-25T12:17:27.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-04-25T12:17:27.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T12:17:27.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-04-25T12:17:27.702 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T12:17:27.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-04-25T12:17:27.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T12:17:27.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T12:17:27.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T12:17:27.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T12:17:27.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T12:17:27.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-04-25T12:17:27.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T12:17:27.703 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T12:17:27.704 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-04-25T12:17:27.704 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-04-25T12:17:27.704 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-04-25T12:17:27.704 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-04-25T12:17:27.704 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T12:17:27.704 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T12:17:27.704 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T12:17:27.705 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T12:17:27.705 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T12:17:27.705 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-04-25T12:17:27.705 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-04-25T12:17:27.705 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-04-25T12:17:27.705 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T12:17:27.705 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T12:17:27.706 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-04-25T12:17:27.706 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-04-25T12:17:27.706 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-04-25T12:17:27.706 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-04-25T12:17:27.706 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T12:17:27.706 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-04-25T12:17:27.706 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T12:17:27.706 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-04-25T12:17:27.707 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 148977, 2024-04-25T12:17:27.707 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 30, 2024-04-25T12:17:27.707 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-04-25T12:17:27.707 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-04-25T12:17:27.707 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-04-25T12:17:27.708 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-04-25T12:17:27.708 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-04-25T12:17:27.708 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-04-25T12:17:27.708 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-04-25T12:17:27.708 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-04-25T12:17:27.708 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-04-25T12:17:27.708 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-04-25T12:17:27.708 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T12:17:27.709 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-04-25T12:17:27.709 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-04-25T12:17:27.709 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-04-25T12:17:27.709 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-04-25T12:17:27.709 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-04-25T12:17:27.709 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-04-25T12:17:27.709 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-04-25T12:17:27.709 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-04-25T12:17:27.709 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-04-25T12:17:27.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-04-25T12:17:27.710 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T12:17:27.710 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-04-25T12:17:27.710 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-04-25T12:17:27.710 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-04-25T12:17:27.710 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 222, 2024-04-25T12:17:27.711 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1558143, 2024-04-25T12:17:27.711 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:27.711 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 222, 2024-04-25T12:17:27.711 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-04-25T12:17:27.711 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-04-25T12:17:27.711 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 222, 2024-04-25T12:17:27.711 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1558143, 2024-04-25T12:17:27.711 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-04-25T12:17:27.712 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-04-25T12:17:27.712 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-04-25T12:17:27.712 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-04-25T12:17:27.712 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:27.712 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-04-25T12:17:27.712 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-04-25T12:17:28.390 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T12:17:28.391 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T12:17:28.391 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T12:17:28.391 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T12:17:28.391 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T12:17:28.391 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T12:17:28.392 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T12:17:28.392 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T12:17:28.392 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T12:17:28.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:28 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:28.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:28 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:28.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:28 smithi019 ceph-mon[29213]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:28.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:28 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:28.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:28 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:28.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:28 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:28.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:28 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:28.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:28 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:17:28.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:28 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:17:28.552 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:28 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:28.644 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:17:28.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:28 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:28.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:28 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:28.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:28 smithi169 ceph-mon[32710]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:28.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:28 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:28.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:28 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:28.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:28 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:28.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:28 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:28.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:28 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:17:28.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:28 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:17:28.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:28 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:28.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:28 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:28.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:28 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:28.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:28 smithi150 ceph-mon[32250]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:28.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:28 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:28.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:28 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:28.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:28 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:28.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:28 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T12:17:28.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:28 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T12:17:28.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:28 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T12:17:28.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:28 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' 2024-04-25T12:17:29.416 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:29 smithi150 ceph-mon[32250]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-25T12:17:29.416 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:29 smithi150 ceph-mon[32250]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T12:17:29.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:29 smithi019 ceph-mon[29213]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-25T12:17:29.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:29 smithi019 ceph-mon[29213]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T12:17:29.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:29 smithi169 ceph-mon[32710]: Adjusting osd_memory_target on smithi019 to 177.2M 2024-04-25T12:17:29.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:29 smithi169 ceph-mon[32710]: Unable to set osd_memory_target on smithi019 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T12:17:30.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:30 smithi150 ceph-mon[32250]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:30.533 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-25T12:17:30.533 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T12:17:30.533 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 86, 2024-04-25T12:17:30.533 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 120, 2024-04-25T12:17:30.534 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 29630, 2024-04-25T12:17:30.534 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 88419, 2024-04-25T12:17:30.534 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 15, 2024-04-25T12:17:30.534 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 5, 2024-04-25T12:17:30.534 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.012619885, 2024-04-25T12:17:30.534 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.002926579, 2024-04-25T12:17:30.534 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.008144962, 2024-04-25T12:17:30.534 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000678750, 2024-04-25T12:17:30.535 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:30.535 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 114, 2024-04-25T12:17:30.535 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.020246066, 2024-04-25T12:17:30.535 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000177597 2024-04-25T12:17:30.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.535 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:30.535 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 153, 2024-04-25T12:17:30.536 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000028790, 2024-04-25T12:17:30.536 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000188 2024-04-25T12:17:30.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.536 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3488, 2024-04-25T12:17:30.536 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 11056 2024-04-25T12:17:30.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.536 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T12:17:30.536 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 635, 2024-04-25T12:17:30.537 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 794, 2024-04-25T12:17:30.537 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1776567, 2024-04-25T12:17:30.537 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 344456, 2024-04-25T12:17:30.537 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 7, 2024-04-25T12:17:30.537 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2024-04-25T12:17:30.537 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.099427340, 2024-04-25T12:17:30.537 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.037397089, 2024-04-25T12:17:30.537 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.061622145, 2024-04-25T12:17:30.538 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.009659146, 2024-04-25T12:17:30.538 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:30.538 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 799, 2024-04-25T12:17:30.538 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.054328929, 2024-04-25T12:17:30.538 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000067996 2024-04-25T12:17:30.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.538 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:30.538 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 888, 2024-04-25T12:17:30.539 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000257289, 2024-04-25T12:17:30.539 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000289 2024-04-25T12:17:30.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.539 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1650960, 2024-04-25T12:17:30.539 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 131472 2024-04-25T12:17:30.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.539 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T12:17:30.539 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 65, 2024-04-25T12:17:30.540 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 218, 2024-04-25T12:17:30.540 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 26584, 2024-04-25T12:17:30.540 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 472103, 2024-04-25T12:17:30.540 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 12, 2024-04-25T12:17:30.540 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 8, 2024-04-25T12:17:30.540 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.015657063, 2024-04-25T12:17:30.540 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.007460162, 2024-04-25T12:17:30.540 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.006460777, 2024-04-25T12:17:30.540 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000271780, 2024-04-25T12:17:30.541 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:30.541 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 214, 2024-04-25T12:17:30.541 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.025012361, 2024-04-25T12:17:30.541 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000116880 2024-04-25T12:17:30.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.541 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:30.541 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 93, 2024-04-25T12:17:30.542 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000016369, 2024-04-25T12:17:30.542 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000176 2024-04-25T12:17:30.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.542 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-25T12:17:30.542 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 441536 2024-04-25T12:17:30.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.542 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-25T12:17:30.542 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-04-25T12:17:30.542 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-25T12:17:30.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.543 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-04-25T12:17:30.543 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-25T12:17:30.543 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-25T12:17:30.543 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 30, 2024-04-25T12:17:30.543 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.002182770, 2024-04-25T12:17:30.543 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000072759 2024-04-25T12:17:30.543 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:30.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.544 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-04-25T12:17:30.544 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-25T12:17:30.544 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-25T12:17:30.544 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2024-04-25T12:17:30.544 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000008195, 2024-04-25T12:17:30.544 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000004097 2024-04-25T12:17:30.544 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:30.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.545 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-04-25T12:17:30.545 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 6, 2024-04-25T12:17:30.545 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 3, 2024-04-25T12:17:30.545 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-04-25T12:17:30.545 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3, 2024-04-25T12:17:30.545 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000544424, 2024-04-25T12:17:30.545 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000181474 2024-04-25T12:17:30.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.545 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-04-25T12:17:30.546 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 3, 2024-04-25T12:17:30.546 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-04-25T12:17:30.546 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-04-25T12:17:30.546 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 11, 2024-04-25T12:17:30.546 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-04-25T12:17:30.546 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-04-25T12:17:30.546 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 14, 2024-04-25T12:17:30.546 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 1, 2024-04-25T12:17:30.546 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 13, 2024-04-25T12:17:30.547 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-04-25T12:17:30.547 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 12, 2024-04-25T12:17:30.547 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-04-25T12:17:30.547 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 1, 2024-04-25T12:17:30.547 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 1, 2024-04-25T12:17:30.547 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 2, 2024-04-25T12:17:30.547 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 12, 2024-04-25T12:17:30.547 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 9, 2024-04-25T12:17:30.547 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-04-25T12:17:30.548 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 2, 2024-04-25T12:17:30.548 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-04-25T12:17:30.548 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-04-25T12:17:30.548 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-04-25T12:17:30.548 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-04-25T12:17:30.548 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-04-25T12:17:30.548 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-04-25T12:17:30.548 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-04-25T12:17:30.548 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-04-25T12:17:30.549 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-04-25T12:17:30.549 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-04-25T12:17:30.549 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-04-25T12:17:30.549 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-04-25T12:17:30.549 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 4, 2024-04-25T12:17:30.549 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 234, 2024-04-25T12:17:30.549 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-04-25T12:17:30.549 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T12:17:30.549 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-04-25T12:17:30.550 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T12:17:30.550 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-04-25T12:17:30.550 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-04-25T12:17:30.550 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T12:17:30.550 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T12:17:30.550 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-04-25T12:17:30.550 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-04-25T12:17:30.550 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T12:17:30.551 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2024-04-25T12:17:30.551 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T12:17:30.551 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T12:17:30.551 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T12:17:30.551 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2024-04-25T12:17:30.551 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2024-04-25T12:17:30.551 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2024-04-25T12:17:30.551 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2024-04-25T12:17:30.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.551 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-04-25T12:17:30.552 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-04-25T12:17:30.552 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 8, 2024-04-25T12:17:30.552 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 5, 2024-04-25T12:17:30.552 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 6, 2024-04-25T12:17:30.552 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-04-25T12:17:30.552 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-04-25T12:17:30.552 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-04-25T12:17:30.552 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-04-25T12:17:30.552 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2024-04-25T12:17:30.553 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2024-04-25T12:17:30.553 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-04-25T12:17:30.553 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-04-25T12:17:30.553 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-04-25T12:17:30.553 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-04-25T12:17:30.553 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-04-25T12:17:30.553 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-04-25T12:17:30.553 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-04-25T12:17:30.554 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-04-25T12:17:30.554 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-04-25T12:17:30.554 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T12:17:30.554 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-04-25T12:17:30.554 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-04-25T12:17:30.554 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-04-25T12:17:30.554 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-04-25T12:17:30.554 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-04-25T12:17:30.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.555 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-04-25T12:17:30.555 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-04-25T12:17:30.555 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 3, 2024-04-25T12:17:30.555 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-04-25T12:17:30.555 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-04-25T12:17:30.555 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 3, 2024-04-25T12:17:30.555 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-04-25T12:17:30.555 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-04-25T12:17:30.556 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 2, 2024-04-25T12:17:30.556 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-04-25T12:17:30.556 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-04-25T12:17:30.556 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 2, 2024-04-25T12:17:30.556 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 2, 2024-04-25T12:17:30.556 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-04-25T12:17:30.556 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-04-25T12:17:30.556 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-04-25T12:17:30.556 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 4195283, 2024-04-25T12:17:30.556 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 0, 2024-04-25T12:17:30.557 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-04-25T12:17:30.557 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2024-04-25T12:17:30.557 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.003941664, 2024-04-25T12:17:30.557 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001970832 2024-04-25T12:17:30.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.557 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 0 2024-04-25T12:17:30.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.557 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-04-25T12:17:30.558 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 17, 2024-04-25T12:17:30.558 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 17, 2024-04-25T12:17:30.558 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 0, 2024-04-25T12:17:30.558 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 16, 2024-04-25T12:17:30.558 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 17, 2024-04-25T12:17:30.558 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 1, 2024-04-25T12:17:30.558 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 14, 2024-04-25T12:17:30.559 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 14, 2024-04-25T12:17:30.559 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 0, 2024-04-25T12:17:30.559 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 1, 2024-04-25T12:17:30.559 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 1, 2024-04-25T12:17:30.559 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 0, 2024-04-25T12:17:30.559 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 52080, 2024-04-25T12:17:30.559 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 223608 2024-04-25T12:17:30.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.559 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-04-25T12:17:30.560 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 6, 2024-04-25T12:17:30.560 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-04-25T12:17:30.560 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 6, 2024-04-25T12:17:30.560 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 8, 2024-04-25T12:17:30.560 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2024-04-25T12:17:30.560 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-04-25T12:17:30.560 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.560 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.560 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.561 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-04-25T12:17:30.561 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3, 2024-04-25T12:17:30.561 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000544424, 2024-04-25T12:17:30.561 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000181474 2024-04-25T12:17:30.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.561 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-04-25T12:17:30.562 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.562 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.562 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.562 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-04-25T12:17:30.562 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.562 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.562 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.563 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-04-25T12:17:30.563 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.563 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.563 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.563 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-04-25T12:17:30.563 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.563 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.564 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.564 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-04-25T12:17:30.564 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.564 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.564 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.564 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-04-25T12:17:30.565 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.565 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.565 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.565 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-04-25T12:17:30.565 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.565 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.565 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.566 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-04-25T12:17:30.566 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.566 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.566 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.566 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-04-25T12:17:30.566 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.567 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.567 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.567 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-04-25T12:17:30.567 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:30 smithi019 ceph-mon[29213]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:30.569 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.569 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.569 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-04-25T12:17:30.569 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.569 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.570 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.570 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-04-25T12:17:30.570 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.570 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.570 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.570 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-04-25T12:17:30.571 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.571 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.571 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.571 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-04-25T12:17:30.571 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.571 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.571 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.572 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-04-25T12:17:30.572 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.572 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.572 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.572 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-04-25T12:17:30.572 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.572 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.573 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.573 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-04-25T12:17:30.573 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.573 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.573 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.573 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-04-25T12:17:30.573 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.574 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.574 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.574 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-04-25T12:17:30.574 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.574 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.574 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.575 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-04-25T12:17:30.575 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.575 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.575 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.575 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-04-25T12:17:30.575 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.575 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.576 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.576 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-04-25T12:17:30.576 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.576 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.576 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.576 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-04-25T12:17:30.576 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.577 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.577 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.577 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-04-25T12:17:30.577 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.577 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.577 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.578 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-04-25T12:17:30.578 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.578 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.578 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.578 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-04-25T12:17:30.578 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.578 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.579 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.579 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-04-25T12:17:30.579 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.579 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.579 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.579 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-04-25T12:17:30.579 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-04-25T12:17:30.580 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-04-25T12:17:30.580 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.580 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.580 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.580 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:30.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.580 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-04-25T12:17:30.580 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 1, 2024-04-25T12:17:30.581 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 1, 2024-04-25T12:17:30.581 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 0, 2024-04-25T12:17:30.581 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 1, 2024-04-25T12:17:30.581 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-04-25T12:17:30.581 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 2, 2024-04-25T12:17:30.581 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 2, 2024-04-25T12:17:30.581 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 567, 2024-04-25T12:17:30.581 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-04-25T12:17:30.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.582 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-25T12:17:30.582 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-25T12:17:30.582 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-25T12:17:30.582 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T12:17:30.582 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-25T12:17:30.582 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T12:17:30.582 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-25T12:17:30.583 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T12:17:30.583 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T12:17:30.583 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T12:17:30.583 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T12:17:30.583 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T12:17:30.583 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-25T12:17:30.583 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T12:17:30.583 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T12:17:30.583 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-25T12:17:30.584 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-25T12:17:30.584 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-25T12:17:30.584 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-25T12:17:30.584 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T12:17:30.584 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T12:17:30.584 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T12:17:30.584 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T12:17:30.584 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T12:17:30.584 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-25T12:17:30.585 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-25T12:17:30.585 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-25T12:17:30.585 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T12:17:30.585 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T12:17:30.585 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-25T12:17:30.585 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-25T12:17:30.585 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-25T12:17:30.585 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-25T12:17:30.585 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T12:17:30.586 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-25T12:17:30.586 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T12:17:30.586 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-25T12:17:30.586 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 256393, 2024-04-25T12:17:30.586 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 96, 2024-04-25T12:17:30.586 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 176, 2024-04-25T12:17:30.586 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 2, 2024-04-25T12:17:30.586 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-25T12:17:30.587 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-25T12:17:30.587 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-25T12:17:30.587 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-25T12:17:30.587 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-25T12:17:30.587 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-25T12:17:30.587 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8224, 2024-04-25T12:17:30.587 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 171, 2024-04-25T12:17:30.587 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T12:17:30.587 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-25T12:17:30.588 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-25T12:17:30.588 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-25T12:17:30.588 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 69408, 2024-04-25T12:17:30.588 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 255, 2024-04-25T12:17:30.588 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-25T12:17:30.588 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-25T12:17:30.588 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-25T12:17:30.588 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-25T12:17:30.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.589 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-04-25T12:17:30.589 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-04-25T12:17:30.589 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-04-25T12:17:30.589 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 37, 2024-04-25T12:17:30.589 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 11191, 2024-04-25T12:17:30.589 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 28, 2024-04-25T12:17:30.589 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 37, 2024-04-25T12:17:30.589 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-04-25T12:17:30.589 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 37, 2024-04-25T12:17:30.590 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.221995769, 2024-04-25T12:17:30.590 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.005999885 2024-04-25T12:17:30.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.590 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-04-25T12:17:30.590 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-04-25T12:17:30.590 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 37, 2024-04-25T12:17:30.590 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 59 2024-04-25T12:17:30.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.590 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 37, 2024-04-25T12:17:30.591 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 0, 2024-04-25T12:17:30.591 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 37, 2024-04-25T12:17:30.591 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-04-25T12:17:30.591 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-04-25T12:17:30.591 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 0, 2024-04-25T12:17:30.591 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 10, 2024-04-25T12:17:30.591 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 0, 2024-04-25T12:17:30.591 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 2, 2024-04-25T12:17:30.591 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 6, 2024-04-25T12:17:30.592 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-04-25T12:17:30.592 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-04-25T12:17:30.592 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-04-25T12:17:30.592 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-04-25T12:17:30.592 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 6, 2024-04-25T12:17:30.592 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-04-25T12:17:30.592 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-04-25T12:17:30.592 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-04-25T12:17:30.593 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 0, 2024-04-25T12:17:30.593 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 20, 2024-04-25T12:17:30.593 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-04-25T12:17:30.593 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-04-25T12:17:30.593 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-04-25T12:17:30.593 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-04-25T12:17:30.593 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-04-25T12:17:30.594 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-04-25T12:17:30.594 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T12:17:30.594 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-04-25T12:17:30.594 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-04-25T12:17:30.594 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-04-25T12:17:30.594 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-04-25T12:17:30.594 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-04-25T12:17:30.594 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-04-25T12:17:30.595 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-04-25T12:17:30.595 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-04-25T12:17:30.595 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-04-25T12:17:30.595 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-04-25T12:17:30.595 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-04-25T12:17:30.595 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-04-25T12:17:30.596 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-04-25T12:17:30.596 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-04-25T12:17:30.596 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-04-25T12:17:30.596 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-04-25T12:17:30.596 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-04-25T12:17:30.597 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-04-25T12:17:30.597 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-04-25T12:17:30.597 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 96, 2024-04-25T12:17:30.597 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-04-25T12:17:30.597 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 11, 2024-04-25T12:17:30.597 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 10, 2024-04-25T12:17:30.598 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 10, 2024-04-25T12:17:30.598 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-04-25T12:17:30.598 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-04-25T12:17:30.598 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 14, 2024-04-25T12:17:30.598 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 0, 2024-04-25T12:17:30.598 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 1 2024-04-25T12:17:30.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.599 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-04-25T12:17:30.599 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 1, 2024-04-25T12:17:30.599 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-04-25T12:17:30.599 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-04-25T12:17:30.599 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-04-25T12:17:30.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.599 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-04-25T12:17:30.600 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-04-25T12:17:30.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.600 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-04-25T12:17:30.600 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-04-25T12:17:30.600 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T12:17:30.600 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-04-25T12:17:30.600 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2024-04-25T12:17:30.600 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 0, 2024-04-25T12:17:30.601 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2024-04-25T12:17:30.601 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-04-25T12:17:30.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.601 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T12:17:30.601 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:30.601 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-25T12:17:30.601 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:30.601 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 782, 2024-04-25T12:17:30.602 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1745453, 2024-04-25T12:17:30.602 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:30.602 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 782, 2024-04-25T12:17:30.602 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T12:17:30.602 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T12:17:30.602 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 782, 2024-04-25T12:17:30.602 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1745453, 2024-04-25T12:17:30.603 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:30.603 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.603 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.603 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.603 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:30.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.604 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-04-25T12:17:30.604 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:30.604 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-25T12:17:30.604 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:30.604 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:30.604 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:30.604 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:30.605 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:30.605 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 37, 2024-04-25T12:17:30.605 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 11191, 2024-04-25T12:17:30.605 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 37, 2024-04-25T12:17:30.605 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 11191, 2024-04-25T12:17:30.605 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:30.605 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.605 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.606 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.606 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:30.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.606 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-04-25T12:17:30.606 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:30.606 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-04-25T12:17:30.606 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:30.607 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:30.607 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:30.607 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:30.607 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:30.607 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 37, 2024-04-25T12:17:30.607 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 37, 2024-04-25T12:17:30.607 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 37, 2024-04-25T12:17:30.608 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 37, 2024-04-25T12:17:30.608 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:30.608 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.608 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.608 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.608 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:30.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.609 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-04-25T12:17:30.609 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:30.609 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-25T12:17:30.609 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:30.609 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:30.609 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:30.609 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:30.609 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:30.609 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T12:17:30.609 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T12:17:30.610 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-04-25T12:17:30.610 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-04-25T12:17:30.610 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:30.610 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.610 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.610 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.610 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:30.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:30.610 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x555a26ff0500": { 2024-04-25T12:17:30.611 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:30.611 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-25T12:17:30.611 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:30.611 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 3, 2024-04-25T12:17:30.611 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 979, 2024-04-25T12:17:30.611 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:30.611 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 3, 2024-04-25T12:17:30.611 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T12:17:30.611 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T12:17:30.612 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2, 2024-04-25T12:17:30.612 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 979, 2024-04-25T12:17:30.612 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:30.612 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:30.612 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:30.612 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:30.612 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:30.612 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:30.613 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-25T12:17:30.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:30 smithi169 ceph-mon[32710]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:31.371 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T12:17:31.371 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T12:17:31.371 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=0 2024-04-25T12:17:31.371 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=8 2024-04-25T12:17:31.371 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=0 2024-04-25T12:17:31.372 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-04-25T12:17:31.372 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T12:17:31.372 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T12:17:31.372 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T12:17:31.620 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:17:32.416 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:32 smithi150 ceph-mon[32250]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:32.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:32 smithi019 ceph-mon[29213]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:32.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:32 smithi169 ceph-mon[32710]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:33.411 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-25T12:17:33.411 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T12:17:33.412 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 794, 2024-04-25T12:17:33.412 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 794, 2024-04-25T12:17:33.412 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 266043, 2024-04-25T12:17:33.412 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 251675, 2024-04-25T12:17:33.412 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2024-04-25T12:17:33.412 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-04-25T12:17:33.412 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.084173423, 2024-04-25T12:17:33.413 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.032676005, 2024-04-25T12:17:33.413 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.019592820, 2024-04-25T12:17:33.413 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.023477659, 2024-04-25T12:17:33.413 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:33.413 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 793, 2024-04-25T12:17:33.413 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.033009252, 2024-04-25T12:17:33.413 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000041625 2024-04-25T12:17:33.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.413 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:33.414 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.414 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.414 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.414 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T12:17:33.414 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T12:17:33.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.415 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T12:17:33.415 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1513, 2024-04-25T12:17:33.415 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 1497, 2024-04-25T12:17:33.415 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1961233, 2024-04-25T12:17:33.415 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 535390, 2024-04-25T12:17:33.415 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-04-25T12:17:33.415 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-04-25T12:17:33.415 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.126751176, 2024-04-25T12:17:33.415 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.035740277, 2024-04-25T12:17:33.416 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.060998655, 2024-04-25T12:17:33.416 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.035380004, 2024-04-25T12:17:33.416 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:33.416 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1496, 2024-04-25T12:17:33.416 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.043869143, 2024-04-25T12:17:33.416 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000029324 2024-04-25T12:17:33.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.416 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:33.416 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.417 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.417 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.417 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1636288, 2024-04-25T12:17:33.417 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 130832 2024-04-25T12:17:33.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.417 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T12:17:33.417 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 3, 2024-04-25T12:17:33.417 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 160, 2024-04-25T12:17:33.418 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 588, 2024-04-25T12:17:33.418 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 440494, 2024-04-25T12:17:33.418 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2024-04-25T12:17:33.418 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2024-04-25T12:17:33.418 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.010166518, 2024-04-25T12:17:33.418 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.006373002, 2024-04-25T12:17:33.418 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001911563, 2024-04-25T12:17:33.418 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000021470, 2024-04-25T12:17:33.419 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:33.419 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 157, 2024-04-25T12:17:33.419 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.012444704, 2024-04-25T12:17:33.419 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000079265 2024-04-25T12:17:33.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.419 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:33.419 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.419 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.419 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.420 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T12:17:33.420 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 440176 2024-04-25T12:17:33.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.420 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-25T12:17:33.420 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-04-25T12:17:33.420 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-25T12:17:33.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.420 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-04-25T12:17:33.421 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-25T12:17:33.421 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-25T12:17:33.421 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3546, 2024-04-25T12:17:33.421 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.084831716, 2024-04-25T12:17:33.421 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000023923 2024-04-25T12:17:33.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:33.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.421 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-04-25T12:17:33.421 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-25T12:17:33.422 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-25T12:17:33.422 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.422 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.422 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.422 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:33.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.422 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-04-25T12:17:33.422 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 0, 2024-04-25T12:17:33.422 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 0, 2024-04-25T12:17:33.423 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-04-25T12:17:33.423 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.423 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.423 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.423 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-04-25T12:17:33.423 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-04-25T12:17:33.423 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-04-25T12:17:33.423 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-04-25T12:17:33.424 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 0, 2024-04-25T12:17:33.424 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-04-25T12:17:33.424 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-04-25T12:17:33.424 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 0, 2024-04-25T12:17:33.424 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 0, 2024-04-25T12:17:33.424 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 0, 2024-04-25T12:17:33.424 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-04-25T12:17:33.424 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 0, 2024-04-25T12:17:33.424 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-04-25T12:17:33.425 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 0, 2024-04-25T12:17:33.425 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 0, 2024-04-25T12:17:33.425 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 1, 2024-04-25T12:17:33.425 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 0, 2024-04-25T12:17:33.425 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 0, 2024-04-25T12:17:33.425 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-04-25T12:17:33.425 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-04-25T12:17:33.425 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-04-25T12:17:33.425 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-04-25T12:17:33.426 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-04-25T12:17:33.426 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-04-25T12:17:33.426 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-04-25T12:17:33.426 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-04-25T12:17:33.426 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-04-25T12:17:33.426 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-04-25T12:17:33.426 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-04-25T12:17:33.426 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-04-25T12:17:33.426 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-04-25T12:17:33.427 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-04-25T12:17:33.427 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 0, 2024-04-25T12:17:33.427 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 0, 2024-04-25T12:17:33.427 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-04-25T12:17:33.427 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T12:17:33.427 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-04-25T12:17:33.427 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T12:17:33.427 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-04-25T12:17:33.428 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-04-25T12:17:33.428 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T12:17:33.428 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T12:17:33.428 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-04-25T12:17:33.428 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-04-25T12:17:33.428 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T12:17:33.428 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2024-04-25T12:17:33.428 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T12:17:33.428 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T12:17:33.428 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T12:17:33.429 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2024-04-25T12:17:33.429 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2024-04-25T12:17:33.429 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2024-04-25T12:17:33.429 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2024-04-25T12:17:33.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.429 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-04-25T12:17:33.429 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-04-25T12:17:33.429 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-04-25T12:17:33.430 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-04-25T12:17:33.430 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-04-25T12:17:33.430 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-04-25T12:17:33.430 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-04-25T12:17:33.430 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-04-25T12:17:33.430 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-04-25T12:17:33.430 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2024-04-25T12:17:33.430 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2024-04-25T12:17:33.431 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-04-25T12:17:33.431 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-04-25T12:17:33.431 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-04-25T12:17:33.431 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-04-25T12:17:33.431 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-04-25T12:17:33.431 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-04-25T12:17:33.431 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-04-25T12:17:33.431 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-04-25T12:17:33.432 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-04-25T12:17:33.432 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T12:17:33.432 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-04-25T12:17:33.432 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-04-25T12:17:33.432 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-04-25T12:17:33.432 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-04-25T12:17:33.432 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-04-25T12:17:33.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.432 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-04-25T12:17:33.433 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-04-25T12:17:33.433 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 0, 2024-04-25T12:17:33.433 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-04-25T12:17:33.433 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-04-25T12:17:33.433 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 3, 2024-04-25T12:17:33.433 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-04-25T12:17:33.433 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-04-25T12:17:33.433 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 0, 2024-04-25T12:17:33.433 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-04-25T12:17:33.434 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-04-25T12:17:33.434 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 2, 2024-04-25T12:17:33.434 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 2, 2024-04-25T12:17:33.434 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-04-25T12:17:33.434 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-04-25T12:17:33.434 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-04-25T12:17:33.434 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 4195283, 2024-04-25T12:17:33.435 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 4194892, 2024-04-25T12:17:33.435 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-04-25T12:17:33.435 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.435 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.435 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.435 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 3 2024-04-25T12:17:33.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.435 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-04-25T12:17:33.436 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 3, 2024-04-25T12:17:33.436 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 3, 2024-04-25T12:17:33.436 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 0, 2024-04-25T12:17:33.436 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 1, 2024-04-25T12:17:33.436 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 1, 2024-04-25T12:17:33.436 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 0, 2024-04-25T12:17:33.436 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 0, 2024-04-25T12:17:33.436 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 0, 2024-04-25T12:17:33.436 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 0, 2024-04-25T12:17:33.437 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 0, 2024-04-25T12:17:33.437 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 0, 2024-04-25T12:17:33.437 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 0, 2024-04-25T12:17:33.437 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 51024, 2024-04-25T12:17:33.437 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 215416 2024-04-25T12:17:33.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.437 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-04-25T12:17:33.437 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 0, 2024-04-25T12:17:33.437 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-04-25T12:17:33.437 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 0, 2024-04-25T12:17:33.438 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 0, 2024-04-25T12:17:33.438 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2024-04-25T12:17:33.438 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-04-25T12:17:33.438 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.438 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.438 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.438 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-04-25T12:17:33.438 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.439 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.439 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.439 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-04-25T12:17:33.439 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.439 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.439 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.439 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-04-25T12:17:33.439 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.440 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.440 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.440 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-04-25T12:17:33.440 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.440 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.440 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.440 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-04-25T12:17:33.441 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.441 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.441 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.441 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-04-25T12:17:33.441 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.441 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.441 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.441 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-04-25T12:17:33.442 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.442 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.442 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.442 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-04-25T12:17:33.442 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.442 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.442 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.443 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-04-25T12:17:33.443 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.443 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.443 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.443 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-04-25T12:17:33.443 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.443 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.443 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.444 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-04-25T12:17:33.444 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.444 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.445 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.445 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-04-25T12:17:33.445 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.445 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.445 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.445 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-04-25T12:17:33.445 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.446 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.446 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.446 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-04-25T12:17:33.446 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.446 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.446 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.447 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-04-25T12:17:33.447 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.447 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.447 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.447 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-04-25T12:17:33.447 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.447 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.447 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.448 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-04-25T12:17:33.448 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.448 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.448 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.448 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-04-25T12:17:33.448 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.448 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.448 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.449 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-04-25T12:17:33.449 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.449 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.449 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.449 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-04-25T12:17:33.449 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.450 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.450 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.450 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-04-25T12:17:33.450 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.450 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.450 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.450 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-04-25T12:17:33.450 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.451 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.451 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.451 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-04-25T12:17:33.451 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.451 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.451 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.451 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-04-25T12:17:33.452 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.452 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.452 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.452 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-04-25T12:17:33.452 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.452 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.452 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.453 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-04-25T12:17:33.453 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.453 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.453 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.453 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-04-25T12:17:33.453 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.453 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.453 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.454 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-04-25T12:17:33.454 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.454 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.454 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.454 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-04-25T12:17:33.454 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-04-25T12:17:33.455 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-04-25T12:17:33.455 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.455 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.455 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:33.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.455 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-04-25T12:17:33.455 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 1, 2024-04-25T12:17:33.455 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 1, 2024-04-25T12:17:33.456 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 0, 2024-04-25T12:17:33.456 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 1, 2024-04-25T12:17:33.456 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-04-25T12:17:33.456 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 0, 2024-04-25T12:17:33.456 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 0, 2024-04-25T12:17:33.456 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 1731, 2024-04-25T12:17:33.456 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-04-25T12:17:33.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.456 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-25T12:17:33.457 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-25T12:17:33.457 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-25T12:17:33.457 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T12:17:33.457 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-25T12:17:33.457 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T12:17:33.457 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-25T12:17:33.457 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T12:17:33.457 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T12:17:33.457 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T12:17:33.458 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T12:17:33.458 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T12:17:33.458 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-25T12:17:33.458 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T12:17:33.458 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T12:17:33.458 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-25T12:17:33.458 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-25T12:17:33.458 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-25T12:17:33.459 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-25T12:17:33.459 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T12:17:33.459 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T12:17:33.459 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T12:17:33.459 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T12:17:33.459 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T12:17:33.459 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-25T12:17:33.459 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-25T12:17:33.459 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-25T12:17:33.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T12:17:33.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T12:17:33.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-25T12:17:33.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-25T12:17:33.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-25T12:17:33.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-25T12:17:33.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T12:17:33.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-25T12:17:33.460 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T12:17:33.461 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-25T12:17:33.461 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 189433, 2024-04-25T12:17:33.461 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 52, 2024-04-25T12:17:33.461 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-04-25T12:17:33.461 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-04-25T12:17:33.461 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-25T12:17:33.461 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-25T12:17:33.461 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-25T12:17:33.461 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-25T12:17:33.462 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-25T12:17:33.462 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-25T12:17:33.462 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8224, 2024-04-25T12:17:33.462 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 171, 2024-04-25T12:17:33.462 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T12:17:33.462 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-25T12:17:33.462 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-25T12:17:33.462 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-25T12:17:33.463 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 8512, 2024-04-25T12:17:33.463 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 13, 2024-04-25T12:17:33.463 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-25T12:17:33.463 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-25T12:17:33.463 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-25T12:17:33.463 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-25T12:17:33.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.463 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-04-25T12:17:33.463 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-04-25T12:17:33.464 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-04-25T12:17:33.464 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 2063, 2024-04-25T12:17:33.464 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 21, 2024-04-25T12:17:33.464 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 3, 2024-04-25T12:17:33.464 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 2063, 2024-04-25T12:17:33.464 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-04-25T12:17:33.464 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2063, 2024-04-25T12:17:33.464 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.958962246, 2024-04-25T12:17:33.465 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000949569 2024-04-25T12:17:33.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.465 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-04-25T12:17:33.465 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-04-25T12:17:33.465 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2063, 2024-04-25T12:17:33.465 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2064 2024-04-25T12:17:33.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.465 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 2063, 2024-04-25T12:17:33.465 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 2063, 2024-04-25T12:17:33.465 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 0, 2024-04-25T12:17:33.466 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-04-25T12:17:33.466 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-04-25T12:17:33.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 1314, 2024-04-25T12:17:33.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 0, 2024-04-25T12:17:33.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 747, 2024-04-25T12:17:33.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 0, 2024-04-25T12:17:33.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 0, 2024-04-25T12:17:33.467 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-04-25T12:17:33.467 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-04-25T12:17:33.467 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-04-25T12:17:33.467 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-04-25T12:17:33.467 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 0, 2024-04-25T12:17:33.467 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-04-25T12:17:33.467 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-04-25T12:17:33.467 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-04-25T12:17:33.467 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 1, 2024-04-25T12:17:33.468 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 0, 2024-04-25T12:17:33.468 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-04-25T12:17:33.468 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-04-25T12:17:33.468 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-04-25T12:17:33.468 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-04-25T12:17:33.468 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-04-25T12:17:33.468 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-04-25T12:17:33.468 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T12:17:33.468 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-04-25T12:17:33.469 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-04-25T12:17:33.469 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-04-25T12:17:33.469 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-04-25T12:17:33.469 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-04-25T12:17:33.469 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-04-25T12:17:33.469 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-04-25T12:17:33.469 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-04-25T12:17:33.469 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-04-25T12:17:33.470 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-04-25T12:17:33.470 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-04-25T12:17:33.470 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-04-25T12:17:33.470 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-04-25T12:17:33.470 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-04-25T12:17:33.470 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-04-25T12:17:33.470 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-04-25T12:17:33.470 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-04-25T12:17:33.471 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-04-25T12:17:33.471 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-04-25T12:17:33.471 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 96, 2024-04-25T12:17:33.471 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-04-25T12:17:33.471 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 11, 2024-04-25T12:17:33.471 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 5, 2024-04-25T12:17:33.471 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 5, 2024-04-25T12:17:33.471 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-04-25T12:17:33.471 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-04-25T12:17:33.471 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 0, 2024-04-25T12:17:33.472 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 2, 2024-04-25T12:17:33.472 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 0 2024-04-25T12:17:33.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.472 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-04-25T12:17:33.472 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 0, 2024-04-25T12:17:33.472 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-04-25T12:17:33.472 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-04-25T12:17:33.472 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-04-25T12:17:33.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.473 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-04-25T12:17:33.473 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-04-25T12:17:33.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.473 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-04-25T12:17:33.473 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-04-25T12:17:33.473 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T12:17:33.473 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-04-25T12:17:33.473 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2024-04-25T12:17:33.473 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 0, 2024-04-25T12:17:33.474 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2024-04-25T12:17:33.474 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-04-25T12:17:33.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.474 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T12:17:33.474 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:33.474 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-25T12:17:33.474 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:33.474 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 2306, 2024-04-25T12:17:33.475 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 2005063, 2024-04-25T12:17:33.475 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:33.475 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 2306, 2024-04-25T12:17:33.475 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T12:17:33.475 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T12:17:33.475 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2306, 2024-04-25T12:17:33.475 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2005063, 2024-04-25T12:17:33.475 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:33.476 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.476 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.476 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:33.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.476 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-04-25T12:17:33.476 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:33.476 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-25T12:17:33.476 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:33.477 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:33.477 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:33.477 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:33.477 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:33.477 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2063, 2024-04-25T12:17:33.477 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 984, 2024-04-25T12:17:33.477 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2063, 2024-04-25T12:17:33.477 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 984, 2024-04-25T12:17:33.477 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:33.478 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.478 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.478 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.478 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:33.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.478 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-04-25T12:17:33.478 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:33.478 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-04-25T12:17:33.479 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:33.479 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:33.479 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:33.479 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:33.479 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:33.479 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2063, 2024-04-25T12:17:33.479 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 2063, 2024-04-25T12:17:33.479 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2063, 2024-04-25T12:17:33.479 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2063, 2024-04-25T12:17:33.479 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:33.480 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.480 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.480 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.480 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:33.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.480 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-04-25T12:17:33.480 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:33.480 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-25T12:17:33.480 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:33.481 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:33.481 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:33.481 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:33.481 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:33.481 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T12:17:33.481 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T12:17:33.481 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-04-25T12:17:33.481 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-04-25T12:17:33.482 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:33.482 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.482 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.482 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.482 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:33.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:33.482 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x55ff71ea46e0": { 2024-04-25T12:17:33.483 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:33.483 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-25T12:17:33.483 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:33.483 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:33.483 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:33.483 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:33.483 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:33.484 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T12:17:33.484 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T12:17:33.484 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-04-25T12:17:33.484 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-04-25T12:17:33.484 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:33.484 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:33.484 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:33.485 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:33.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:33.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:33.485 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-25T12:17:34.261 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T12:17:34.261 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T12:17:34.262 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-25T12:17:34.262 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-25T12:17:34.262 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-25T12:17:34.262 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-25T12:17:34.262 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T12:17:34.262 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T12:17:34.262 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T12:17:34.503 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:17:34.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:34 smithi019 ceph-mon[29213]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:34.625 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:34 smithi150 ceph-mon[32250]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:34.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:34 smithi169 ceph-mon[32710]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:36.338 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:36 smithi150 ceph-mon[32250]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:36.360 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-25T12:17:36.361 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T12:17:36.361 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1537, 2024-04-25T12:17:36.361 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 1539, 2024-04-25T12:17:36.361 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 547323, 2024-04-25T12:17:36.361 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 488776, 2024-04-25T12:17:36.361 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2024-04-25T12:17:36.361 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-04-25T12:17:36.361 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.141739692, 2024-04-25T12:17:36.361 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.056414738, 2024-04-25T12:17:36.362 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.031975531, 2024-04-25T12:17:36.362 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.039982314, 2024-04-25T12:17:36.362 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:36.362 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1537, 2024-04-25T12:17:36.362 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.056251211, 2024-04-25T12:17:36.362 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000036598 2024-04-25T12:17:36.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.362 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:36.363 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.363 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.363 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.363 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T12:17:36.363 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T12:17:36.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.363 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T12:17:36.364 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 10, 2024-04-25T12:17:36.364 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 165, 2024-04-25T12:17:36.364 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 4968, 2024-04-25T12:17:36.364 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 445194, 2024-04-25T12:17:36.364 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 7, 2024-04-25T12:17:36.364 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 5, 2024-04-25T12:17:36.364 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.011134241, 2024-04-25T12:17:36.364 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.006692492, 2024-04-25T12:17:36.364 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.002476988, 2024-04-25T12:17:36.365 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000078902, 2024-04-25T12:17:36.365 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:36.365 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 163, 2024-04-25T12:17:36.365 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.015357994, 2024-04-25T12:17:36.365 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000094220 2024-04-25T12:17:36.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.366 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:36.366 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.366 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.366 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.366 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-25T12:17:36.366 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 443856 2024-04-25T12:17:36.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.367 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T12:17:36.367 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 968, 2024-04-25T12:17:36.367 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 952, 2024-04-25T12:17:36.367 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1815515, 2024-04-25T12:17:36.367 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 361539, 2024-04-25T12:17:36.367 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-04-25T12:17:36.367 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-04-25T12:17:36.367 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.106388179, 2024-04-25T12:17:36.368 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.031414919, 2024-04-25T12:17:36.368 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.051219867, 2024-04-25T12:17:36.368 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.028687559, 2024-04-25T12:17:36.368 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:36.368 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 951, 2024-04-25T12:17:36.368 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.030874869, 2024-04-25T12:17:36.368 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000032465 2024-04-25T12:17:36.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.368 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:36.369 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.369 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.369 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.369 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1636176, 2024-04-25T12:17:36.369 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 130352 2024-04-25T12:17:36.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.369 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-25T12:17:36.369 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-04-25T12:17:36.370 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-25T12:17:36.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.370 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-04-25T12:17:36.370 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-25T12:17:36.370 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-25T12:17:36.370 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3799, 2024-04-25T12:17:36.370 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.097020541, 2024-04-25T12:17:36.370 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000025538 2024-04-25T12:17:36.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:36.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.371 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-04-25T12:17:36.371 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-25T12:17:36.371 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-25T12:17:36.371 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.371 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.371 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:36.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.372 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-04-25T12:17:36.372 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 0, 2024-04-25T12:17:36.372 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 0, 2024-04-25T12:17:36.372 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-04-25T12:17:36.372 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.372 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.372 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.373 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-04-25T12:17:36.373 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-04-25T12:17:36.373 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-04-25T12:17:36.373 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-04-25T12:17:36.373 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 0, 2024-04-25T12:17:36.373 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-04-25T12:17:36.373 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-04-25T12:17:36.373 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 12, 2024-04-25T12:17:36.374 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 0, 2024-04-25T12:17:36.374 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 12, 2024-04-25T12:17:36.374 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-04-25T12:17:36.374 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 12, 2024-04-25T12:17:36.374 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-04-25T12:17:36.374 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 0, 2024-04-25T12:17:36.374 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 0, 2024-04-25T12:17:36.374 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 2, 2024-04-25T12:17:36.374 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 0, 2024-04-25T12:17:36.375 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 0, 2024-04-25T12:17:36.375 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-04-25T12:17:36.375 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-04-25T12:17:36.375 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-04-25T12:17:36.375 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-04-25T12:17:36.375 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-04-25T12:17:36.375 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-04-25T12:17:36.375 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-04-25T12:17:36.376 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-04-25T12:17:36.376 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-04-25T12:17:36.376 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-04-25T12:17:36.376 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-04-25T12:17:36.376 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-04-25T12:17:36.376 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-04-25T12:17:36.376 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-04-25T12:17:36.376 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 4, 2024-04-25T12:17:36.376 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 234, 2024-04-25T12:17:36.377 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 6, 2024-04-25T12:17:36.377 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T12:17:36.377 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-04-25T12:17:36.377 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T12:17:36.377 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-04-25T12:17:36.377 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-04-25T12:17:36.377 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T12:17:36.377 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T12:17:36.377 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-04-25T12:17:36.378 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-04-25T12:17:36.378 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T12:17:36.378 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2024-04-25T12:17:36.378 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T12:17:36.378 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T12:17:36.378 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T12:17:36.378 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2024-04-25T12:17:36.378 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2024-04-25T12:17:36.379 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2024-04-25T12:17:36.379 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2024-04-25T12:17:36.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.379 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-04-25T12:17:36.379 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-04-25T12:17:36.379 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-04-25T12:17:36.379 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-04-25T12:17:36.379 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-04-25T12:17:36.380 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-04-25T12:17:36.380 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-04-25T12:17:36.380 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-04-25T12:17:36.380 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-04-25T12:17:36.380 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2024-04-25T12:17:36.380 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2024-04-25T12:17:36.380 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-04-25T12:17:36.380 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-04-25T12:17:36.380 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-04-25T12:17:36.381 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-04-25T12:17:36.381 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-04-25T12:17:36.381 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-04-25T12:17:36.381 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-04-25T12:17:36.381 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-04-25T12:17:36.381 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-04-25T12:17:36.381 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T12:17:36.381 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-04-25T12:17:36.381 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-04-25T12:17:36.382 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-04-25T12:17:36.382 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-04-25T12:17:36.382 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-04-25T12:17:36.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.382 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-04-25T12:17:36.382 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-04-25T12:17:36.382 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 0, 2024-04-25T12:17:36.382 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-04-25T12:17:36.383 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-04-25T12:17:36.383 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 100, 2024-04-25T12:17:36.383 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-04-25T12:17:36.383 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-04-25T12:17:36.383 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 0, 2024-04-25T12:17:36.383 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-04-25T12:17:36.383 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-04-25T12:17:36.383 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 2, 2024-04-25T12:17:36.383 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 2, 2024-04-25T12:17:36.384 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-04-25T12:17:36.384 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-04-25T12:17:36.384 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-04-25T12:17:36.384 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 4283934, 2024-04-25T12:17:36.384 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 4283822, 2024-04-25T12:17:36.384 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-04-25T12:17:36.384 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.384 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.385 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.385 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 100 2024-04-25T12:17:36.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.385 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-04-25T12:17:36.385 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 13, 2024-04-25T12:17:36.385 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 13, 2024-04-25T12:17:36.385 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 0, 2024-04-25T12:17:36.386 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 8, 2024-04-25T12:17:36.386 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 8, 2024-04-25T12:17:36.386 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 0, 2024-04-25T12:17:36.386 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 12, 2024-04-25T12:17:36.386 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 12, 2024-04-25T12:17:36.386 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 0, 2024-04-25T12:17:36.386 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 0, 2024-04-25T12:17:36.386 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 0, 2024-04-25T12:17:36.386 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 0, 2024-04-25T12:17:36.387 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 50540, 2024-04-25T12:17:36.387 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 215416 2024-04-25T12:17:36.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.387 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-04-25T12:17:36.387 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 0, 2024-04-25T12:17:36.387 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-04-25T12:17:36.387 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 0, 2024-04-25T12:17:36.387 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 0, 2024-04-25T12:17:36.387 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2024-04-25T12:17:36.388 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-04-25T12:17:36.388 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.388 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.388 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.388 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-04-25T12:17:36.388 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.388 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.389 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.389 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-04-25T12:17:36.389 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.389 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.389 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.389 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-04-25T12:17:36.389 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.390 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.390 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.390 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-04-25T12:17:36.390 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.390 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.390 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.391 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-04-25T12:17:36.391 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.391 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.391 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.391 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-04-25T12:17:36.391 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.391 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.392 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.392 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-04-25T12:17:36.392 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.392 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.392 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.392 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-04-25T12:17:36.392 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.393 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.393 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.393 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-04-25T12:17:36.393 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.393 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.393 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.393 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-04-25T12:17:36.394 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.394 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.394 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.394 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-04-25T12:17:36.394 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.395 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.395 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.395 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-04-25T12:17:36.396 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.396 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.396 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.396 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-04-25T12:17:36.396 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.396 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.396 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.397 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-04-25T12:17:36.397 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.397 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.397 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.397 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-04-25T12:17:36.397 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.397 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.398 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.398 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-04-25T12:17:36.398 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.398 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.398 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.398 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-04-25T12:17:36.399 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.399 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.399 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.399 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-04-25T12:17:36.399 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.399 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.399 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.400 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-04-25T12:17:36.400 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.400 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.400 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.400 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-04-25T12:17:36.400 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.400 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.401 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.401 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-04-25T12:17:36.401 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.401 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.401 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.401 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-04-25T12:17:36.402 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.402 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.402 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.402 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-04-25T12:17:36.402 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.402 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.402 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-04-25T12:17:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-04-25T12:17:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.403 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.404 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.404 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-04-25T12:17:36.404 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.404 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.404 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.404 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-04-25T12:17:36.404 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.405 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.405 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.405 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-04-25T12:17:36.405 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.405 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.405 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.406 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-04-25T12:17:36.406 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-04-25T12:17:36.406 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-04-25T12:17:36.406 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.406 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.406 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.406 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:36.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.406 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-04-25T12:17:36.407 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 2, 2024-04-25T12:17:36.407 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 5, 2024-04-25T12:17:36.407 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 3, 2024-04-25T12:17:36.407 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 4, 2024-04-25T12:17:36.407 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-04-25T12:17:36.407 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 0, 2024-04-25T12:17:36.407 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 0, 2024-04-25T12:17:36.407 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 2838, 2024-04-25T12:17:36.408 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-04-25T12:17:36.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.408 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-25T12:17:36.408 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-25T12:17:36.408 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-25T12:17:36.408 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T12:17:36.408 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-25T12:17:36.408 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T12:17:36.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-25T12:17:36.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T12:17:36.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T12:17:36.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T12:17:36.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T12:17:36.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T12:17:36.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-25T12:17:36.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T12:17:36.409 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T12:17:36.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-25T12:17:36.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-25T12:17:36.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-25T12:17:36.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-25T12:17:36.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T12:17:36.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T12:17:36.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T12:17:36.410 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T12:17:36.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T12:17:36.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-25T12:17:36.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-25T12:17:36.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-25T12:17:36.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T12:17:36.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T12:17:36.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-25T12:17:36.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-25T12:17:36.411 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-25T12:17:36.412 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-25T12:17:36.412 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T12:17:36.412 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-25T12:17:36.412 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T12:17:36.412 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-25T12:17:36.412 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 197513, 2024-04-25T12:17:36.412 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 56, 2024-04-25T12:17:36.412 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-04-25T12:17:36.413 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-04-25T12:17:36.413 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-25T12:17:36.413 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-25T12:17:36.413 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-25T12:17:36.413 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-25T12:17:36.413 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-25T12:17:36.413 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-25T12:17:36.413 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8224, 2024-04-25T12:17:36.413 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 171, 2024-04-25T12:17:36.414 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T12:17:36.414 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-25T12:17:36.414 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-25T12:17:36.414 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-25T12:17:36.414 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 47746, 2024-04-25T12:17:36.414 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 220, 2024-04-25T12:17:36.414 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-25T12:17:36.414 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-25T12:17:36.415 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-25T12:17:36.415 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-25T12:17:36.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.415 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-04-25T12:17:36.415 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-04-25T12:17:36.415 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-04-25T12:17:36.415 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 2268, 2024-04-25T12:17:36.415 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 26, 2024-04-25T12:17:36.415 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 4, 2024-04-25T12:17:36.416 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 2268, 2024-04-25T12:17:36.416 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-04-25T12:17:36.416 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2268, 2024-04-25T12:17:36.416 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.732966634, 2024-04-25T12:17:36.416 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000764094 2024-04-25T12:17:36.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.416 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-04-25T12:17:36.416 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-04-25T12:17:36.417 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2268, 2024-04-25T12:17:36.417 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2269 2024-04-25T12:17:36.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.417 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 2268, 2024-04-25T12:17:36.417 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 2268, 2024-04-25T12:17:36.417 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 0, 2024-04-25T12:17:36.417 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-04-25T12:17:36.417 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-04-25T12:17:36.417 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 1471, 2024-04-25T12:17:36.418 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 0, 2024-04-25T12:17:36.418 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 794, 2024-04-25T12:17:36.418 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 0, 2024-04-25T12:17:36.418 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 0, 2024-04-25T12:17:36.418 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-04-25T12:17:36.418 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-04-25T12:17:36.418 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-04-25T12:17:36.418 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-04-25T12:17:36.419 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 0, 2024-04-25T12:17:36.419 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-04-25T12:17:36.419 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-04-25T12:17:36.419 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-04-25T12:17:36.419 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 2, 2024-04-25T12:17:36.419 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 0, 2024-04-25T12:17:36.419 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-04-25T12:17:36.419 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-04-25T12:17:36.419 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-04-25T12:17:36.420 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-04-25T12:17:36.420 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-04-25T12:17:36.420 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-04-25T12:17:36.420 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T12:17:36.420 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-04-25T12:17:36.420 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-04-25T12:17:36.420 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-04-25T12:17:36.420 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-04-25T12:17:36.420 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-04-25T12:17:36.421 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-04-25T12:17:36.421 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-04-25T12:17:36.421 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-04-25T12:17:36.421 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-04-25T12:17:36.421 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-04-25T12:17:36.421 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-04-25T12:17:36.421 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-04-25T12:17:36.421 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-04-25T12:17:36.422 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-04-25T12:17:36.422 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-04-25T12:17:36.422 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-04-25T12:17:36.422 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-04-25T12:17:36.422 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-04-25T12:17:36.422 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-04-25T12:17:36.422 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 96, 2024-04-25T12:17:36.422 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-04-25T12:17:36.423 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 11, 2024-04-25T12:17:36.423 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 6, 2024-04-25T12:17:36.423 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 6, 2024-04-25T12:17:36.423 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-04-25T12:17:36.423 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-04-25T12:17:36.423 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 0, 2024-04-25T12:17:36.423 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 2, 2024-04-25T12:17:36.423 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 0 2024-04-25T12:17:36.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.424 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-04-25T12:17:36.424 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 0, 2024-04-25T12:17:36.424 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-04-25T12:17:36.424 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-04-25T12:17:36.424 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-04-25T12:17:36.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.424 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-04-25T12:17:36.424 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-04-25T12:17:36.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.425 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-04-25T12:17:36.425 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-04-25T12:17:36.425 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T12:17:36.425 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-04-25T12:17:36.425 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2024-04-25T12:17:36.425 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 0, 2024-04-25T12:17:36.425 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2024-04-25T12:17:36.425 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-04-25T12:17:36.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.426 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T12:17:36.426 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:36.426 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-25T12:17:36.426 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:36.426 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 2511, 2024-04-25T12:17:36.426 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 2126561, 2024-04-25T12:17:36.426 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:36.427 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 2511, 2024-04-25T12:17:36.427 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T12:17:36.427 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T12:17:36.427 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2511, 2024-04-25T12:17:36.427 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2126561, 2024-04-25T12:17:36.427 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:36.427 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.427 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.428 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:36.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.428 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-04-25T12:17:36.428 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:36.428 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-25T12:17:36.428 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:36.428 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:36.428 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:36.429 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:36.429 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:36.429 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2268, 2024-04-25T12:17:36.429 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 89640, 2024-04-25T12:17:36.429 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2268, 2024-04-25T12:17:36.429 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 89640, 2024-04-25T12:17:36.429 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:36.429 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.430 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.430 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.430 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:36.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.430 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-04-25T12:17:36.430 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:36.430 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-04-25T12:17:36.430 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:36.431 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:36.431 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:36.431 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:36.431 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:36.431 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2268, 2024-04-25T12:17:36.431 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 2268, 2024-04-25T12:17:36.431 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2268, 2024-04-25T12:17:36.431 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2268, 2024-04-25T12:17:36.431 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:36.432 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.432 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.432 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:36.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.432 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-04-25T12:17:36.432 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:36.432 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-25T12:17:36.433 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:36.433 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:36.433 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:36.433 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:36.433 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:36.433 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T12:17:36.433 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T12:17:36.433 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-04-25T12:17:36.433 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-04-25T12:17:36.434 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:36.434 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.434 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.434 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.434 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:36.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:36.434 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x561a3483c6e0": { 2024-04-25T12:17:36.434 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:36.435 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-25T12:17:36.435 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:36.435 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:36.435 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:36.435 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:36.435 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:36.435 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T12:17:36.435 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T12:17:36.435 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-04-25T12:17:36.436 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-04-25T12:17:36.436 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:36.436 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:36.436 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:36.436 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:36.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:36.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:36.436 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-25T12:17:36.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:36 smithi019 ceph-mon[29213]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:36.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:36 smithi169 ceph-mon[32710]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:37.173 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T12:17:37.173 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T12:17:37.173 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-25T12:17:37.173 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-25T12:17:37.173 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-25T12:17:37.173 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T12:17:37.173 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T12:17:37.174 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T12:17:37.174 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T12:17:37.414 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T12:17:38.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:38 smithi019 ceph-mon[29213]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:38.637 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:38 smithi150 ceph-mon[32250]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:38.658 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:38 smithi169 ceph-mon[32710]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:39.213 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-25T12:17:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T12:17:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1501, 2024-04-25T12:17:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 1503, 2024-04-25T12:17:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 450529, 2024-04-25T12:17:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 477319, 2024-04-25T12:17:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2024-04-25T12:17:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-04-25T12:17:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.140978221, 2024-04-25T12:17:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.049462368, 2024-04-25T12:17:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.035875309, 2024-04-25T12:17:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.041905281, 2024-04-25T12:17:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1501, 2024-04-25T12:17:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.056905929, 2024-04-25T12:17:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000037912 2024-04-25T12:17:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T12:17:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T12:17:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T12:17:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 636, 2024-04-25T12:17:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 796, 2024-04-25T12:17:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 156936, 2024-04-25T12:17:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 644000, 2024-04-25T12:17:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 7, 2024-04-25T12:17:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 5, 2024-04-25T12:17:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.071949089, 2024-04-25T12:17:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.020745699, 2024-04-25T12:17:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.018122069, 2024-04-25T12:17:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.026570477, 2024-04-25T12:17:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 793, 2024-04-25T12:17:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.030815033, 2024-04-25T12:17:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000038858 2024-04-25T12:17:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T12:17:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 442320 2024-04-25T12:17:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T12:17:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 249, 2024-04-25T12:17:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 232, 2024-04-25T12:17:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1640636, 2024-04-25T12:17:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 134158, 2024-04-25T12:17:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 3, 2024-04-25T12:17:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-04-25T12:17:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.051357113, 2024-04-25T12:17:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.014673804, 2024-04-25T12:17:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.040877497, 2024-04-25T12:17:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.005768154, 2024-04-25T12:17:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 232, 2024-04-25T12:17:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.014887769, 2024-04-25T12:17:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000064171 2024-04-25T12:17:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1640368, 2024-04-25T12:17:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 133840 2024-04-25T12:17:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-25T12:17:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-04-25T12:17:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-25T12:17:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-04-25T12:17:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-25T12:17:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-25T12:17:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3405, 2024-04-25T12:17:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.088120646, 2024-04-25T12:17:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000025879 2024-04-25T12:17:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-04-25T12:17:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-25T12:17:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-25T12:17:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-04-25T12:17:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 0, 2024-04-25T12:17:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 0, 2024-04-25T12:17:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-04-25T12:17:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-04-25T12:17:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-04-25T12:17:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-04-25T12:17:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-04-25T12:17:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 0, 2024-04-25T12:17:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-04-25T12:17:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-04-25T12:17:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 0, 2024-04-25T12:17:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 0, 2024-04-25T12:17:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 0, 2024-04-25T12:17:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-04-25T12:17:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 0, 2024-04-25T12:17:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-04-25T12:17:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 0, 2024-04-25T12:17:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 0, 2024-04-25T12:17:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 1, 2024-04-25T12:17:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 0, 2024-04-25T12:17:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 0, 2024-04-25T12:17:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-04-25T12:17:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-04-25T12:17:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-04-25T12:17:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-04-25T12:17:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-04-25T12:17:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-04-25T12:17:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-04-25T12:17:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-04-25T12:17:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-04-25T12:17:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-04-25T12:17:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-04-25T12:17:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-04-25T12:17:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-04-25T12:17:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-04-25T12:17:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 0, 2024-04-25T12:17:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 0, 2024-04-25T12:17:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-04-25T12:17:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T12:17:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-04-25T12:17:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T12:17:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-04-25T12:17:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-04-25T12:17:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T12:17:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T12:17:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-04-25T12:17:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-04-25T12:17:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T12:17:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2024-04-25T12:17:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T12:17:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T12:17:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T12:17:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2024-04-25T12:17:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2024-04-25T12:17:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2024-04-25T12:17:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2024-04-25T12:17:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-04-25T12:17:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-04-25T12:17:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-04-25T12:17:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-04-25T12:17:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-04-25T12:17:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-04-25T12:17:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-04-25T12:17:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-04-25T12:17:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-04-25T12:17:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2024-04-25T12:17:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2024-04-25T12:17:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-04-25T12:17:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-04-25T12:17:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-04-25T12:17:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-04-25T12:17:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-04-25T12:17:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-04-25T12:17:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-04-25T12:17:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-04-25T12:17:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-04-25T12:17:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T12:17:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-04-25T12:17:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-04-25T12:17:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-04-25T12:17:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-04-25T12:17:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-04-25T12:17:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-04-25T12:17:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-04-25T12:17:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 0, 2024-04-25T12:17:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-04-25T12:17:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-04-25T12:17:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 7, 2024-04-25T12:17:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-04-25T12:17:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-04-25T12:17:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 0, 2024-04-25T12:17:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-04-25T12:17:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-04-25T12:17:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 2, 2024-04-25T12:17:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 2, 2024-04-25T12:17:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-04-25T12:17:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-04-25T12:17:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-04-25T12:17:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 4196264, 2024-04-25T12:17:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 4196123, 2024-04-25T12:17:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-04-25T12:17:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 7 2024-04-25T12:17:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-04-25T12:17:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 3, 2024-04-25T12:17:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 3, 2024-04-25T12:17:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 0, 2024-04-25T12:17:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 1, 2024-04-25T12:17:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 1, 2024-04-25T12:17:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 0, 2024-04-25T12:17:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 0, 2024-04-25T12:17:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 0, 2024-04-25T12:17:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 0, 2024-04-25T12:17:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 0, 2024-04-25T12:17:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 0, 2024-04-25T12:17:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 0, 2024-04-25T12:17:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 53432, 2024-04-25T12:17:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 215416 2024-04-25T12:17:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-04-25T12:17:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 0, 2024-04-25T12:17:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-04-25T12:17:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 0, 2024-04-25T12:17:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 0, 2024-04-25T12:17:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2024-04-25T12:17:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-04-25T12:17:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-04-25T12:17:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-04-25T12:17:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-04-25T12:17:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-04-25T12:17:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-04-25T12:17:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-04-25T12:17:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-04-25T12:17:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-04-25T12:17:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-04-25T12:17:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-04-25T12:17:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-04-25T12:17:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-04-25T12:17:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-04-25T12:17:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-04-25T12:17:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-04-25T12:17:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-04-25T12:17:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-04-25T12:17:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-04-25T12:17:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-04-25T12:17:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-04-25T12:17:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.252 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.252 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.252 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-04-25T12:17:39.252 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.252 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.252 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.253 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-04-25T12:17:39.253 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.253 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.253 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.253 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-04-25T12:17:39.253 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.253 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.253 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.254 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-04-25T12:17:39.254 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.254 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.254 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.254 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-04-25T12:17:39.254 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.255 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.255 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.255 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-04-25T12:17:39.255 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.255 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.255 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.256 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-04-25T12:17:39.256 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.256 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.256 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.256 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-04-25T12:17:39.256 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.256 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.256 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.257 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-04-25T12:17:39.257 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-04-25T12:17:39.257 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-04-25T12:17:39.257 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.257 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.257 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.257 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:39.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.258 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-04-25T12:17:39.258 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 1, 2024-04-25T12:17:39.258 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 3, 2024-04-25T12:17:39.258 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 2, 2024-04-25T12:17:39.258 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 3, 2024-04-25T12:17:39.258 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-04-25T12:17:39.258 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 0, 2024-04-25T12:17:39.258 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 0, 2024-04-25T12:17:39.259 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 3952, 2024-04-25T12:17:39.259 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-04-25T12:17:39.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.259 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-25T12:17:39.259 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-25T12:17:39.259 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-25T12:17:39.259 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T12:17:39.259 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-25T12:17:39.259 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T12:17:39.260 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-25T12:17:39.260 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T12:17:39.260 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T12:17:39.260 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T12:17:39.260 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T12:17:39.260 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T12:17:39.260 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-25T12:17:39.260 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T12:17:39.260 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T12:17:39.261 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-25T12:17:39.261 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-25T12:17:39.261 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-25T12:17:39.261 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-25T12:17:39.261 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T12:17:39.261 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T12:17:39.261 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T12:17:39.261 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T12:17:39.262 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T12:17:39.262 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-25T12:17:39.262 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-25T12:17:39.262 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-25T12:17:39.262 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T12:17:39.262 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T12:17:39.262 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-25T12:17:39.262 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-25T12:17:39.263 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-25T12:17:39.263 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-25T12:17:39.263 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T12:17:39.263 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-25T12:17:39.263 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T12:17:39.263 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-25T12:17:39.263 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 189393, 2024-04-25T12:17:39.263 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 51, 2024-04-25T12:17:39.264 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-04-25T12:17:39.264 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-04-25T12:17:39.264 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-25T12:17:39.264 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-25T12:17:39.264 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-25T12:17:39.264 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-25T12:17:39.264 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-25T12:17:39.264 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-25T12:17:39.265 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8224, 2024-04-25T12:17:39.265 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 171, 2024-04-25T12:17:39.265 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T12:17:39.265 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-25T12:17:39.265 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-25T12:17:39.265 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-25T12:17:39.265 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 8512, 2024-04-25T12:17:39.265 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 13, 2024-04-25T12:17:39.265 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-25T12:17:39.266 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-25T12:17:39.266 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-25T12:17:39.266 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-25T12:17:39.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.266 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-04-25T12:17:39.266 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-04-25T12:17:39.266 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-04-25T12:17:39.266 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 2136, 2024-04-25T12:17:39.267 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 21, 2024-04-25T12:17:39.267 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 3, 2024-04-25T12:17:39.267 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 2136, 2024-04-25T12:17:39.267 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-04-25T12:17:39.267 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2136, 2024-04-25T12:17:39.267 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.050979616, 2024-04-25T12:17:39.267 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000492031 2024-04-25T12:17:39.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.267 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-04-25T12:17:39.268 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-04-25T12:17:39.268 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2136, 2024-04-25T12:17:39.268 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2137 2024-04-25T12:17:39.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.268 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 2136, 2024-04-25T12:17:39.268 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 2136, 2024-04-25T12:17:39.268 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 0, 2024-04-25T12:17:39.268 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-04-25T12:17:39.268 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-04-25T12:17:39.269 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 1379, 2024-04-25T12:17:39.269 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 0, 2024-04-25T12:17:39.269 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 755, 2024-04-25T12:17:39.269 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 0, 2024-04-25T12:17:39.269 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 0, 2024-04-25T12:17:39.269 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-04-25T12:17:39.269 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-04-25T12:17:39.269 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-04-25T12:17:39.269 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-04-25T12:17:39.270 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 0, 2024-04-25T12:17:39.270 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-04-25T12:17:39.270 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-04-25T12:17:39.270 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-04-25T12:17:39.270 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 1, 2024-04-25T12:17:39.270 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 0, 2024-04-25T12:17:39.270 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-04-25T12:17:39.271 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-04-25T12:17:39.271 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-04-25T12:17:39.271 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-04-25T12:17:39.271 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-04-25T12:17:39.271 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-04-25T12:17:39.271 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T12:17:39.271 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-04-25T12:17:39.271 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-04-25T12:17:39.271 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-04-25T12:17:39.272 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-04-25T12:17:39.272 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-04-25T12:17:39.272 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-04-25T12:17:39.272 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-04-25T12:17:39.272 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-04-25T12:17:39.272 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-04-25T12:17:39.272 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-04-25T12:17:39.272 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-04-25T12:17:39.272 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-04-25T12:17:39.273 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-04-25T12:17:39.273 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-04-25T12:17:39.273 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-04-25T12:17:39.273 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-04-25T12:17:39.273 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-04-25T12:17:39.273 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-04-25T12:17:39.273 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-04-25T12:17:39.273 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 96, 2024-04-25T12:17:39.274 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-04-25T12:17:39.274 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 11, 2024-04-25T12:17:39.274 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 5, 2024-04-25T12:17:39.274 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 5, 2024-04-25T12:17:39.274 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-04-25T12:17:39.274 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-04-25T12:17:39.274 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 0, 2024-04-25T12:17:39.274 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 2, 2024-04-25T12:17:39.275 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 0 2024-04-25T12:17:39.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.275 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-04-25T12:17:39.275 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 0, 2024-04-25T12:17:39.275 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-04-25T12:17:39.275 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-04-25T12:17:39.275 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-04-25T12:17:39.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.276 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-04-25T12:17:39.276 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-04-25T12:17:39.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.276 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-04-25T12:17:39.276 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-04-25T12:17:39.276 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T12:17:39.276 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-04-25T12:17:39.276 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2024-04-25T12:17:39.276 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 0, 2024-04-25T12:17:39.277 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2024-04-25T12:17:39.277 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-04-25T12:17:39.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.277 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T12:17:39.277 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:39.277 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-25T12:17:39.277 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:39.277 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 2382, 2024-04-25T12:17:39.278 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 2018252, 2024-04-25T12:17:39.278 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:39.278 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 2382, 2024-04-25T12:17:39.278 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T12:17:39.278 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T12:17:39.278 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2382, 2024-04-25T12:17:39.278 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2018252, 2024-04-25T12:17:39.278 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:39.278 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.279 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.279 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.279 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:39.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.279 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-04-25T12:17:39.279 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:39.279 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-25T12:17:39.279 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:39.279 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:39.280 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:39.280 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:39.280 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:39.280 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2136, 2024-04-25T12:17:39.280 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 1965, 2024-04-25T12:17:39.280 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2136, 2024-04-25T12:17:39.280 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1965, 2024-04-25T12:17:39.280 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:39.280 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.281 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.281 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.281 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:39.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.281 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-04-25T12:17:39.281 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:39.281 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-04-25T12:17:39.281 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:39.281 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:39.282 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:39.282 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:39.282 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:39.282 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2136, 2024-04-25T12:17:39.282 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 2136, 2024-04-25T12:17:39.282 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2136, 2024-04-25T12:17:39.282 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2136, 2024-04-25T12:17:39.282 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:39.283 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.283 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.283 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.283 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:39.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.283 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-04-25T12:17:39.283 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:39.283 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-25T12:17:39.284 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:39.284 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:39.284 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:39.284 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:39.284 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:39.284 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T12:17:39.284 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T12:17:39.284 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-04-25T12:17:39.284 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-04-25T12:17:39.285 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:39.285 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.285 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.285 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.285 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:39.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-25T12:17:39.285 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x562d6295a6e0": { 2024-04-25T12:17:39.285 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-25T12:17:39.286 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-25T12:17:39.286 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-25T12:17:39.286 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-25T12:17:39.286 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-25T12:17:39.286 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:39.286 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:39.286 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-25T12:17:39.286 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-25T12:17:39.286 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-04-25T12:17:39.287 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-04-25T12:17:39.287 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-25T12:17:39.287 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-25T12:17:39.287 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-25T12:17:39.287 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-25T12:17:39.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:39.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-25T12:17:39.287 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-25T12:17:40.153 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-25T12:17:40.153 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-25T12:17:40.153 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-25T12:17:40.153 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-04-25T12:17:40.153 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-25T12:17:40.153 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-25T12:17:40.154 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T12:17:40.154 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T12:17:40.154 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T12:17:40.409 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:17:40.416 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:40 smithi150 ceph-mon[32250]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:40.531 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:40 smithi169 ceph-mon[32710]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:40.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:40 smithi019 ceph-mon[29213]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:42.217 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T12:17:42.217 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T12:17:42.217 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 337, 2024-04-25T12:17:42.218 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 645, 2024-04-25T12:17:42.218 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 108790, 2024-04-25T12:17:42.218 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 445882, 2024-04-25T12:17:42.218 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 10, 2024-04-25T12:17:42.218 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-04-25T12:17:42.218 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.062823521, 2024-04-25T12:17:42.218 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.031938759, 2024-04-25T12:17:42.218 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.021726012, 2024-04-25T12:17:42.219 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.004009208, 2024-04-25T12:17:42.219 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:42.219 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 648, 2024-04-25T12:17:42.219 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.053802876, 2024-04-25T12:17:42.219 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000083029 2024-04-25T12:17:42.219 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.219 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:42.219 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 791, 2024-04-25T12:17:42.219 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000208944, 2024-04-25T12:17:42.220 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000264 2024-04-25T12:17:42.220 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.220 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-25T12:17:42.220 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 315792 2024-04-25T12:17:42.220 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.220 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T12:17:42.220 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 117, 2024-04-25T12:17:42.220 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 146, 2024-04-25T12:17:42.221 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 46132, 2024-04-25T12:17:42.221 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 184423, 2024-04-25T12:17:42.221 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 10, 2024-04-25T12:17:42.221 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 4, 2024-04-25T12:17:42.221 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.013634795, 2024-04-25T12:17:42.221 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.005234100, 2024-04-25T12:17:42.221 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.008255470, 2024-04-25T12:17:42.221 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000368164, 2024-04-25T12:17:42.221 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:42.222 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 145, 2024-04-25T12:17:42.222 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.018456778, 2024-04-25T12:17:42.222 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000127288 2024-04-25T12:17:42.222 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.222 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:42.222 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 183, 2024-04-25T12:17:42.222 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000034991, 2024-04-25T12:17:42.222 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000191 2024-04-25T12:17:42.222 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.223 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T12:17:42.223 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 129856 2024-04-25T12:17:42.223 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.223 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T12:17:42.223 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 343, 2024-04-25T12:17:42.223 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 333, 2024-04-25T12:17:42.223 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1675777, 2024-04-25T12:17:42.223 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 211395, 2024-04-25T12:17:42.223 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 13, 2024-04-25T12:17:42.224 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 7, 2024-04-25T12:17:42.224 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.056001485, 2024-04-25T12:17:42.224 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.016654018, 2024-04-25T12:17:42.224 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.043455517, 2024-04-25T12:17:42.224 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.005891515, 2024-04-25T12:17:42.224 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:42.224 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 326, 2024-04-25T12:17:42.224 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.023667178, 2024-04-25T12:17:42.224 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000072598 2024-04-25T12:17:42.225 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.225 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:42.225 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 126, 2024-04-25T12:17:42.225 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000024998, 2024-04-25T12:17:42.225 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000198 2024-04-25T12:17:42.225 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.225 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 1653008, 2024-04-25T12:17:42.226 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 140560 2024-04-25T12:17:42.226 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.226 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T12:17:42.226 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-25T12:17:42.226 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T12:17:42.226 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.226 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-25T12:17:42.226 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T12:17:42.226 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T12:17:42.227 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 48, 2024-04-25T12:17:42.227 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.002895141, 2024-04-25T12:17:42.227 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000060315 2024-04-25T12:17:42.227 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:42.227 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.227 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-25T12:17:42.227 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T12:17:42.227 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T12:17:42.227 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T12:17:42.227 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000006844, 2024-04-25T12:17:42.228 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000003422 2024-04-25T12:17:42.228 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:42.228 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.228 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-25T12:17:42.228 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 6, 2024-04-25T12:17:42.228 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 3, 2024-04-25T12:17:42.228 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-25T12:17:42.228 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3, 2024-04-25T12:17:42.228 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000586890, 2024-04-25T12:17:42.229 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000195630 2024-04-25T12:17:42.229 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.229 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-25T12:17:42.229 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 3, 2024-04-25T12:17:42.229 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-25T12:17:42.229 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-25T12:17:42.229 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 11, 2024-04-25T12:17:42.229 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-04-25T12:17:42.229 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-25T12:17:42.230 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 14, 2024-04-25T12:17:42.230 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 1, 2024-04-25T12:17:42.230 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 13, 2024-04-25T12:17:42.230 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-04-25T12:17:42.230 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 12, 2024-04-25T12:17:42.230 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-04-25T12:17:42.230 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 0, 2024-04-25T12:17:42.230 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 0, 2024-04-25T12:17:42.230 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 2, 2024-04-25T12:17:42.231 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 13, 2024-04-25T12:17:42.231 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 8, 2024-04-25T12:17:42.231 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 1, 2024-04-25T12:17:42.231 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 2, 2024-04-25T12:17:42.231 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-25T12:17:42.231 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-25T12:17:42.231 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-25T12:17:42.231 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-04-25T12:17:42.231 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-25T12:17:42.232 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-04-25T12:17:42.232 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-04-25T12:17:42.232 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-04-25T12:17:42.232 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-04-25T12:17:42.232 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-25T12:17:42.232 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-25T12:17:42.232 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-25T12:17:42.232 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 4, 2024-04-25T12:17:42.232 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 234, 2024-04-25T12:17:42.233 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-04-25T12:17:42.233 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T12:17:42.233 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-25T12:17:42.233 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T12:17:42.233 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-25T12:17:42.233 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-25T12:17:42.233 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T12:17:42.233 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T12:17:42.233 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-25T12:17:42.234 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-04-25T12:17:42.234 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T12:17:42.234 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 0, 2024-04-25T12:17:42.234 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T12:17:42.234 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T12:17:42.234 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T12:17:42.234 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 0, 2024-04-25T12:17:42.234 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-04-25T12:17:42.235 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-04-25T12:17:42.235 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 2 2024-04-25T12:17:42.235 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.235 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-25T12:17:42.235 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-04-25T12:17:42.235 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 6, 2024-04-25T12:17:42.235 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 4, 2024-04-25T12:17:42.235 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 5, 2024-04-25T12:17:42.235 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-04-25T12:17:42.235 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-04-25T12:17:42.236 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-25T12:17:42.236 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-25T12:17:42.236 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-04-25T12:17:42.236 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-04-25T12:17:42.236 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-25T12:17:42.236 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-25T12:17:42.236 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-25T12:17:42.236 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-25T12:17:42.236 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-25T12:17:42.237 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-25T12:17:42.237 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-25T12:17:42.237 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-04-25T12:17:42.237 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-04-25T12:17:42.237 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T12:17:42.237 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-04-25T12:17:42.237 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-25T12:17:42.237 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-04-25T12:17:42.237 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-25T12:17:42.237 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-25T12:17:42.238 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.238 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-25T12:17:42.238 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-25T12:17:42.238 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 6, 2024-04-25T12:17:42.238 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-04-25T12:17:42.238 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-04-25T12:17:42.238 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 6, 2024-04-25T12:17:42.238 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-25T12:17:42.238 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-04-25T12:17:42.239 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 2, 2024-04-25T12:17:42.239 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-04-25T12:17:42.239 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-04-25T12:17:42.239 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 2, 2024-04-25T12:17:42.239 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-04-25T12:17:42.239 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-25T12:17:42.239 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-04-25T12:17:42.239 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-04-25T12:17:42.239 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 4195956, 2024-04-25T12:17:42.239 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 0, 2024-04-25T12:17:42.240 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-25T12:17:42.240 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 5, 2024-04-25T12:17:42.240 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.011054350, 2024-04-25T12:17:42.240 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.002210870 2024-04-25T12:17:42.240 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.240 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 0 2024-04-25T12:17:42.240 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.240 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-25T12:17:42.241 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 17, 2024-04-25T12:17:42.241 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 17, 2024-04-25T12:17:42.241 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 0, 2024-04-25T12:17:42.241 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 16, 2024-04-25T12:17:42.241 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 16, 2024-04-25T12:17:42.241 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 0, 2024-04-25T12:17:42.241 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 14, 2024-04-25T12:17:42.241 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 14, 2024-04-25T12:17:42.241 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 0, 2024-04-25T12:17:42.241 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 0, 2024-04-25T12:17:42.242 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 3, 2024-04-25T12:17:42.242 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 3, 2024-04-25T12:17:42.242 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 54860, 2024-04-25T12:17:42.242 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 223608 2024-04-25T12:17:42.242 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.242 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-25T12:17:42.242 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 6, 2024-04-25T12:17:42.242 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-04-25T12:17:42.242 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 6, 2024-04-25T12:17:42.243 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 8, 2024-04-25T12:17:42.243 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 0, 2024-04-25T12:17:42.243 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-25T12:17:42.243 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.243 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.243 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.243 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.243 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-25T12:17:42.243 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T12:17:42.244 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000425251, 2024-04-25T12:17:42.244 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000212625 2024-04-25T12:17:42.244 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.244 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-25T12:17:42.244 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.244 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.244 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.244 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.244 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-25T12:17:42.245 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.245 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.245 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.245 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.245 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-25T12:17:42.245 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-04-25T12:17:42.245 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000161639, 2024-04-25T12:17:42.245 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000161639 2024-04-25T12:17:42.245 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.246 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-25T12:17:42.246 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.246 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.246 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.246 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.246 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-25T12:17:42.246 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.246 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.247 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.247 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.247 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-25T12:17:42.247 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.247 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.247 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.247 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.247 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-25T12:17:42.247 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.247 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.248 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.248 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.248 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-25T12:17:42.248 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.248 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.248 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.248 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.248 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-25T12:17:42.248 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.249 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.249 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.249 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.249 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-25T12:17:42.249 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.250 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.250 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.250 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.250 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-25T12:17:42.250 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.250 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.250 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.250 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.251 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-25T12:17:42.251 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.251 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.251 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.251 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.251 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-25T12:17:42.251 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.251 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.252 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.252 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.252 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-25T12:17:42.252 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.252 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.252 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.252 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.252 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-25T12:17:42.252 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.252 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.253 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.253 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.253 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-25T12:17:42.253 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.253 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.253 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.253 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.253 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-25T12:17:42.253 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.254 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.254 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.254 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.254 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-25T12:17:42.254 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.254 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.254 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.254 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.254 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-25T12:17:42.255 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.255 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.255 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.255 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.255 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-25T12:17:42.255 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.255 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.255 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.255 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.256 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-25T12:17:42.256 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.256 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.256 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.256 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.256 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-25T12:17:42.256 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.256 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.256 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.257 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.257 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-25T12:17:42.257 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.257 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.257 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.257 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.257 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-25T12:17:42.257 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.257 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.257 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.258 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.258 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-25T12:17:42.258 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.258 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.258 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.258 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.258 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-25T12:17:42.258 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.258 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.259 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.259 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.259 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-25T12:17:42.259 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.259 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.259 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.259 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.259 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-25T12:17:42.259 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-25T12:17:42.260 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-25T12:17:42.260 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.260 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.260 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.260 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:42.260 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.260 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-25T12:17:42.260 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 0, 2024-04-25T12:17:42.260 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 2, 2024-04-25T12:17:42.260 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 2, 2024-04-25T12:17:42.261 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 0, 2024-04-25T12:17:42.261 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-25T12:17:42.261 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 2, 2024-04-25T12:17:42.261 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 2, 2024-04-25T12:17:42.261 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 89, 2024-04-25T12:17:42.261 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-25T12:17:42.261 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.261 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T12:17:42.261 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T12:17:42.262 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T12:17:42.262 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T12:17:42.262 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T12:17:42.262 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T12:17:42.262 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T12:17:42.262 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T12:17:42.262 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T12:17:42.262 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T12:17:42.262 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T12:17:42.263 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T12:17:42.263 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T12:17:42.263 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T12:17:42.263 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T12:17:42.263 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T12:17:42.263 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T12:17:42.263 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T12:17:42.263 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T12:17:42.263 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T12:17:42.264 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T12:17:42.264 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T12:17:42.264 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T12:17:42.264 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T12:17:42.264 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T12:17:42.264 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T12:17:42.264 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T12:17:42.264 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T12:17:42.264 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T12:17:42.265 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T12:17:42.265 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T12:17:42.265 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T12:17:42.265 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T12:17:42.265 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T12:17:42.265 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T12:17:42.265 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T12:17:42.265 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T12:17:42.266 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 255703, 2024-04-25T12:17:42.266 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 91, 2024-04-25T12:17:42.266 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-25T12:17:42.266 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-25T12:17:42.266 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T12:17:42.266 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T12:17:42.266 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T12:17:42.266 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T12:17:42.266 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T12:17:42.267 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T12:17:42.267 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 8224, 2024-04-25T12:17:42.267 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 171, 2024-04-25T12:17:42.267 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T12:17:42.267 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T12:17:42.267 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T12:17:42.267 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T12:17:42.267 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 69080, 2024-04-25T12:17:42.267 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 258, 2024-04-25T12:17:42.268 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T12:17:42.268 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T12:17:42.268 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T12:17:42.268 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T12:17:42.268 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.268 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-25T12:17:42.268 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-25T12:17:42.268 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-25T12:17:42.268 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 44, 2024-04-25T12:17:42.269 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 12189, 2024-04-25T12:17:42.269 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 28, 2024-04-25T12:17:42.269 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 44, 2024-04-25T12:17:42.269 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-25T12:17:42.269 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 44, 2024-04-25T12:17:42.269 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.233995058, 2024-04-25T12:17:42.269 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.005318069 2024-04-25T12:17:42.269 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.269 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-25T12:17:42.270 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-25T12:17:42.270 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 44, 2024-04-25T12:17:42.270 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 68 2024-04-25T12:17:42.270 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.270 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 44, 2024-04-25T12:17:42.270 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 0, 2024-04-25T12:17:42.270 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 44, 2024-04-25T12:17:42.270 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-25T12:17:42.270 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-25T12:17:42.271 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 0, 2024-04-25T12:17:42.271 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 10, 2024-04-25T12:17:42.271 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 0, 2024-04-25T12:17:42.271 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 5, 2024-04-25T12:17:42.271 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 8, 2024-04-25T12:17:42.271 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-25T12:17:42.271 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-25T12:17:42.271 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-25T12:17:42.271 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-04-25T12:17:42.272 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 6, 2024-04-25T12:17:42.272 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-25T12:17:42.272 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-25T12:17:42.272 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-25T12:17:42.272 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 0, 2024-04-25T12:17:42.272 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 20, 2024-04-25T12:17:42.272 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-25T12:17:42.272 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-25T12:17:42.272 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-25T12:17:42.273 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-25T12:17:42.273 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-25T12:17:42.273 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-25T12:17:42.273 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T12:17:42.273 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-25T12:17:42.273 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-25T12:17:42.273 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-04-25T12:17:42.273 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-25T12:17:42.273 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-25T12:17:42.274 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-25T12:17:42.274 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-25T12:17:42.274 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-25T12:17:42.274 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-25T12:17:42.274 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-25T12:17:42.274 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-25T12:17:42.274 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-25T12:17:42.274 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-25T12:17:42.275 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-25T12:17:42.275 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-25T12:17:42.275 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-25T12:17:42.275 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-25T12:17:42.275 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-25T12:17:42.275 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-25T12:17:42.275 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 96, 2024-04-25T12:17:42.275 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-25T12:17:42.275 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 11, 2024-04-25T12:17:42.276 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 8, 2024-04-25T12:17:42.276 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 8, 2024-04-25T12:17:42.276 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-25T12:17:42.276 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-25T12:17:42.276 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 17, 2024-04-25T12:17:42.276 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 0, 2024-04-25T12:17:42.276 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 2 2024-04-25T12:17:42.276 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.276 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-25T12:17:42.276 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 1, 2024-04-25T12:17:42.277 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-25T12:17:42.277 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-25T12:17:42.277 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-25T12:17:42.277 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.277 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-25T12:17:42.277 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-04-25T12:17:42.277 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.277 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-25T12:17:42.277 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-25T12:17:42.278 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T12:17:42.278 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-25T12:17:42.278 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-04-25T12:17:42.278 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-04-25T12:17:42.278 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-04-25T12:17:42.278 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-25T12:17:42.278 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.278 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T12:17:42.278 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:42.279 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T12:17:42.279 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:42.279 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 793, 2024-04-25T12:17:42.279 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1742078, 2024-04-25T12:17:42.279 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:42.279 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 793, 2024-04-25T12:17:42.279 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T12:17:42.279 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T12:17:42.279 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 793, 2024-04-25T12:17:42.279 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1742078, 2024-04-25T12:17:42.280 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:42.280 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.280 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.280 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.280 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:42.280 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.280 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-25T12:17:42.280 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:42.280 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T12:17:42.280 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:42.281 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T12:17:42.281 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T12:17:42.281 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:42.281 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:42.281 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 44, 2024-04-25T12:17:42.281 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 12189, 2024-04-25T12:17:42.281 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 44, 2024-04-25T12:17:42.281 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 12189, 2024-04-25T12:17:42.281 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-25T12:17:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:42.282 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-25T12:17:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T12:17:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T12:17:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 44, 2024-04-25T12:17:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 44, 2024-04-25T12:17:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 44, 2024-04-25T12:17:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 44, 2024-04-25T12:17:42.283 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:42.284 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.284 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.284 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.284 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:42.284 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.284 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-25T12:17:42.284 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:42.284 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T12:17:42.284 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:42.285 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T12:17:42.285 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T12:17:42.285 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:42.285 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:42.285 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T12:17:42.285 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T12:17:42.285 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-25T12:17:42.285 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-25T12:17:42.285 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:42.286 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.286 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.286 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.286 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:42.286 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:42.286 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x55e9f88f0500": { 2024-04-25T12:17:42.286 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:42.286 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T12:17:42.286 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:42.287 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 6, 2024-04-25T12:17:42.287 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1652, 2024-04-25T12:17:42.287 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:42.287 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 6, 2024-04-25T12:17:42.287 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T12:17:42.287 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T12:17:42.287 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 5, 2024-04-25T12:17:42.287 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1652, 2024-04-25T12:17:42.287 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:42.288 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:42.288 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:42.288 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:42.288 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:42.288 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:42.288 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T12:17:42.409 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:42 smithi169 ceph-mon[32710]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:42.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:42 smithi019 ceph-mon[29213]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:42.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:42 smithi150 ceph-mon[32250]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:42.956 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T12:17:42.957 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T12:17:42.957 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T12:17:42.957 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=6 2024-04-25T12:17:42.957 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-25T12:17:42.957 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T12:17:42.957 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T12:17:42.957 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T12:17:42.958 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T12:17:43.195 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:17:44.544 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:44 smithi169 ceph-mon[32710]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:44.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:44 smithi019 ceph-mon[29213]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:44.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:44 smithi150 ceph-mon[32250]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:44.828 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T12:17:44.829 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T12:17:44.829 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 317, 2024-04-25T12:17:44.829 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 315, 2024-04-25T12:17:44.829 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1657557, 2024-04-25T12:17:44.829 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 203079, 2024-04-25T12:17:44.829 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 14, 2024-04-25T12:17:44.829 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 2, 2024-04-25T12:17:44.829 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.064552059, 2024-04-25T12:17:44.830 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.019193074, 2024-04-25T12:17:44.830 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.047517573, 2024-04-25T12:17:44.830 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.006182520, 2024-04-25T12:17:44.830 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:44.830 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 315, 2024-04-25T12:17:44.830 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.024721195, 2024-04-25T12:17:44.830 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000078479 2024-04-25T12:17:44.830 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.831 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:44.831 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 53, 2024-04-25T12:17:44.831 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000011324, 2024-04-25T12:17:44.831 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000213 2024-04-25T12:17:44.831 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.831 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 1651920, 2024-04-25T12:17:44.831 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 145824 2024-04-25T12:17:44.831 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.831 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T12:17:44.831 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 119, 2024-04-25T12:17:44.832 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 152, 2024-04-25T12:17:44.832 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 46553, 2024-04-25T12:17:44.832 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 178853, 2024-04-25T12:17:44.832 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 11, 2024-04-25T12:17:44.832 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 5, 2024-04-25T12:17:44.832 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.013321615, 2024-04-25T12:17:44.832 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.005040702, 2024-04-25T12:17:44.832 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.008264012, 2024-04-25T12:17:44.832 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.000208633, 2024-04-25T12:17:44.833 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:44.833 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 149, 2024-04-25T12:17:44.833 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.020166591, 2024-04-25T12:17:44.833 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000135346 2024-04-25T12:17:44.833 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.833 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:44.833 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 183, 2024-04-25T12:17:44.833 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000038893, 2024-04-25T12:17:44.834 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000212 2024-04-25T12:17:44.834 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.834 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-04-25T12:17:44.834 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 131008 2024-04-25T12:17:44.834 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.834 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T12:17:44.834 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 370, 2024-04-25T12:17:44.834 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 687, 2024-04-25T12:17:44.834 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 123533, 2024-04-25T12:17:44.835 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 575242, 2024-04-25T12:17:44.835 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 9, 2024-04-25T12:17:44.835 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 6, 2024-04-25T12:17:44.835 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.064242876, 2024-04-25T12:17:44.835 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.032884160, 2024-04-25T12:17:44.835 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.024725160, 2024-04-25T12:17:44.835 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.003875078, 2024-04-25T12:17:44.835 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:44.836 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 685, 2024-04-25T12:17:44.836 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.052821202, 2024-04-25T12:17:44.836 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000077111 2024-04-25T12:17:44.836 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.836 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:44.836 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 880, 2024-04-25T12:17:44.836 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000240558, 2024-04-25T12:17:44.836 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000273 2024-04-25T12:17:44.836 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.837 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T12:17:44.837 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 315344 2024-04-25T12:17:44.837 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.837 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T12:17:44.837 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-25T12:17:44.837 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T12:17:44.837 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.837 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-25T12:17:44.837 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T12:17:44.838 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T12:17:44.838 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 54, 2024-04-25T12:17:44.838 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001398974, 2024-04-25T12:17:44.838 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000025906 2024-04-25T12:17:44.838 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:44.838 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.838 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-25T12:17:44.838 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T12:17:44.838 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T12:17:44.839 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T12:17:44.839 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000006030, 2024-04-25T12:17:44.839 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000003015 2024-04-25T12:17:44.839 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:44.839 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.839 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-25T12:17:44.839 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 5, 2024-04-25T12:17:44.839 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 3, 2024-04-25T12:17:44.839 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-25T12:17:44.839 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3, 2024-04-25T12:17:44.840 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001756049, 2024-04-25T12:17:44.840 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000585349 2024-04-25T12:17:44.840 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.840 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-25T12:17:44.840 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 2, 2024-04-25T12:17:44.840 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-25T12:17:44.840 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-25T12:17:44.840 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 11, 2024-04-25T12:17:44.841 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-04-25T12:17:44.841 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-25T12:17:44.841 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 14, 2024-04-25T12:17:44.841 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 1, 2024-04-25T12:17:44.841 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 13, 2024-04-25T12:17:44.841 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-04-25T12:17:44.841 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 12, 2024-04-25T12:17:44.841 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 1, 2024-04-25T12:17:44.841 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 0, 2024-04-25T12:17:44.841 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 0, 2024-04-25T12:17:44.842 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 2, 2024-04-25T12:17:44.842 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 13, 2024-04-25T12:17:44.842 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 6, 2024-04-25T12:17:44.842 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 2, 2024-04-25T12:17:44.842 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 3, 2024-04-25T12:17:44.842 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-25T12:17:44.842 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-25T12:17:44.842 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-25T12:17:44.842 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-04-25T12:17:44.843 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-25T12:17:44.843 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-04-25T12:17:44.843 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-04-25T12:17:44.843 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-04-25T12:17:44.843 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-04-25T12:17:44.843 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-25T12:17:44.843 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-25T12:17:44.843 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-25T12:17:44.843 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 4, 2024-04-25T12:17:44.843 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 234, 2024-04-25T12:17:44.844 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-04-25T12:17:44.844 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T12:17:44.844 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-25T12:17:44.844 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T12:17:44.844 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-25T12:17:44.844 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-25T12:17:44.844 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T12:17:44.844 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T12:17:44.844 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-25T12:17:44.845 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-04-25T12:17:44.845 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T12:17:44.845 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 0, 2024-04-25T12:17:44.845 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T12:17:44.845 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T12:17:44.845 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T12:17:44.845 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 0, 2024-04-25T12:17:44.845 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-04-25T12:17:44.845 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-04-25T12:17:44.846 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 0 2024-04-25T12:17:44.846 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.846 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-25T12:17:44.846 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-04-25T12:17:44.846 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 6, 2024-04-25T12:17:44.846 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 4, 2024-04-25T12:17:44.846 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 6, 2024-04-25T12:17:44.846 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-04-25T12:17:44.846 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-04-25T12:17:44.847 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-25T12:17:44.847 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-25T12:17:44.847 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-04-25T12:17:44.847 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-04-25T12:17:44.847 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-25T12:17:44.847 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-25T12:17:44.847 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-25T12:17:44.847 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-25T12:17:44.847 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-25T12:17:44.848 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-25T12:17:44.848 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-25T12:17:44.848 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-04-25T12:17:44.848 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-04-25T12:17:44.848 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T12:17:44.848 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-04-25T12:17:44.848 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-25T12:17:44.848 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-04-25T12:17:44.849 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-25T12:17:44.849 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-25T12:17:44.849 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.849 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-25T12:17:44.849 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-25T12:17:44.849 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 7, 2024-04-25T12:17:44.849 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-04-25T12:17:44.849 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-04-25T12:17:44.850 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 7, 2024-04-25T12:17:44.850 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-25T12:17:44.850 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-04-25T12:17:44.850 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 2, 2024-04-25T12:17:44.850 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-04-25T12:17:44.850 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-04-25T12:17:44.850 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 2, 2024-04-25T12:17:44.850 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-04-25T12:17:44.850 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-25T12:17:44.851 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-04-25T12:17:44.851 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-04-25T12:17:44.851 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 4196264, 2024-04-25T12:17:44.851 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 0, 2024-04-25T12:17:44.851 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-25T12:17:44.851 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 6, 2024-04-25T12:17:44.851 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.009502583, 2024-04-25T12:17:44.851 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001583763 2024-04-25T12:17:44.851 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.852 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 0 2024-04-25T12:17:44.852 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.852 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-25T12:17:44.852 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 17, 2024-04-25T12:17:44.852 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 17, 2024-04-25T12:17:44.852 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 0, 2024-04-25T12:17:44.852 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 16, 2024-04-25T12:17:44.852 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 16, 2024-04-25T12:17:44.852 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 0, 2024-04-25T12:17:44.852 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 14, 2024-04-25T12:17:44.853 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 15, 2024-04-25T12:17:44.853 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 1, 2024-04-25T12:17:44.853 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 0, 2024-04-25T12:17:44.853 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 2, 2024-04-25T12:17:44.853 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 2, 2024-04-25T12:17:44.853 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 53016, 2024-04-25T12:17:44.853 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 223608 2024-04-25T12:17:44.853 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.853 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-25T12:17:44.854 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 9, 2024-04-25T12:17:44.854 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-04-25T12:17:44.854 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 5, 2024-04-25T12:17:44.854 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 17, 2024-04-25T12:17:44.854 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 1, 2024-04-25T12:17:44.854 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-25T12:17:44.854 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.854 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.854 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.855 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.855 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-25T12:17:44.855 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-04-25T12:17:44.855 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001191722, 2024-04-25T12:17:44.855 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001191722 2024-04-25T12:17:44.855 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.855 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-25T12:17:44.855 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.855 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.856 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.856 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.856 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-25T12:17:44.856 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.856 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.856 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.856 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.856 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-25T12:17:44.856 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T12:17:44.857 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000564327, 2024-04-25T12:17:44.857 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000282163 2024-04-25T12:17:44.857 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.857 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-25T12:17:44.857 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.857 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.857 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.857 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.857 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-25T12:17:44.857 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.858 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.858 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.858 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.858 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-25T12:17:44.858 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.858 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.858 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.858 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.858 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-25T12:17:44.859 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.859 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.859 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.859 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.859 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-25T12:17:44.859 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.859 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.859 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.859 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.860 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-25T12:17:44.860 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.860 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.860 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.860 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.860 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-25T12:17:44.860 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.861 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.861 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.861 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.861 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-25T12:17:44.861 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.861 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.862 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.862 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.862 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-25T12:17:44.862 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.862 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.862 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.862 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.862 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-25T12:17:44.862 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.863 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.863 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.863 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.863 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-25T12:17:44.863 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.863 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.863 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.863 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.863 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-25T12:17:44.864 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.864 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.864 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.864 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.864 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-25T12:17:44.864 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.864 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.864 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.864 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.864 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-25T12:17:44.865 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.865 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.865 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.865 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.865 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-25T12:17:44.865 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.865 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.865 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.865 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.866 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-25T12:17:44.866 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.866 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.866 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.866 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.866 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-25T12:17:44.866 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.866 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.866 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.867 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.867 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-25T12:17:44.867 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.867 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.867 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.867 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.867 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-25T12:17:44.867 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.867 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.868 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.868 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.868 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-25T12:17:44.868 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.868 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.868 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.868 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.868 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-25T12:17:44.868 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.869 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.869 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.869 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.869 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-25T12:17:44.869 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.869 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.869 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.869 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.869 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-25T12:17:44.869 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.870 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.870 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.870 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.870 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-25T12:17:44.870 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.870 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.870 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.870 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.870 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-25T12:17:44.871 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-25T12:17:44.871 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-25T12:17:44.871 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.871 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.871 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.871 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:44.871 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.871 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-25T12:17:44.872 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 1, 2024-04-25T12:17:44.872 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 3, 2024-04-25T12:17:44.872 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 2, 2024-04-25T12:17:44.872 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 1, 2024-04-25T12:17:44.872 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-25T12:17:44.872 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 2, 2024-04-25T12:17:44.872 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 0, 2024-04-25T12:17:44.872 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 633, 2024-04-25T12:17:44.872 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-25T12:17:44.872 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.873 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T12:17:44.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T12:17:44.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T12:17:44.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T12:17:44.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T12:17:44.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T12:17:44.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T12:17:44.873 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T12:17:44.874 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T12:17:44.874 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T12:17:44.874 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T12:17:44.874 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T12:17:44.874 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T12:17:44.874 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T12:17:44.874 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T12:17:44.874 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T12:17:44.874 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T12:17:44.874 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T12:17:44.875 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T12:17:44.875 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T12:17:44.875 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T12:17:44.875 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T12:17:44.875 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T12:17:44.875 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T12:17:44.875 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T12:17:44.875 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T12:17:44.875 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T12:17:44.876 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T12:17:44.876 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T12:17:44.876 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T12:17:44.876 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T12:17:44.876 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T12:17:44.876 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T12:17:44.876 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T12:17:44.876 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T12:17:44.877 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T12:17:44.877 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T12:17:44.877 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 271671, 2024-04-25T12:17:44.877 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 97, 2024-04-25T12:17:44.877 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-25T12:17:44.877 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-25T12:17:44.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T12:17:44.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T12:17:44.877 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T12:17:44.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T12:17:44.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T12:17:44.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T12:17:44.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 8224, 2024-04-25T12:17:44.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 171, 2024-04-25T12:17:44.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T12:17:44.878 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T12:17:44.878 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T12:17:44.879 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T12:17:44.879 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 69120, 2024-04-25T12:17:44.879 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 263, 2024-04-25T12:17:44.879 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T12:17:44.879 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T12:17:44.879 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T12:17:44.879 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T12:17:44.879 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.879 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-25T12:17:44.880 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-25T12:17:44.880 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-25T12:17:44.880 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 44, 2024-04-25T12:17:44.880 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 12407, 2024-04-25T12:17:44.880 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 31, 2024-04-25T12:17:44.880 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 44, 2024-04-25T12:17:44.880 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-25T12:17:44.880 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 44, 2024-04-25T12:17:44.880 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.209995576, 2024-04-25T12:17:44.881 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.004772626 2024-04-25T12:17:44.881 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.881 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-25T12:17:44.881 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-25T12:17:44.881 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 44, 2024-04-25T12:17:44.881 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 68 2024-04-25T12:17:44.881 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.881 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 44, 2024-04-25T12:17:44.881 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 0, 2024-04-25T12:17:44.882 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 44, 2024-04-25T12:17:44.882 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-25T12:17:44.882 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-25T12:17:44.882 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 0, 2024-04-25T12:17:44.882 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 10, 2024-04-25T12:17:44.882 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 0, 2024-04-25T12:17:44.882 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 6, 2024-04-25T12:17:44.882 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 7, 2024-04-25T12:17:44.882 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-25T12:17:44.883 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-25T12:17:44.883 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-25T12:17:44.883 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-04-25T12:17:44.883 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 6, 2024-04-25T12:17:44.883 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-25T12:17:44.883 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-25T12:17:44.883 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-25T12:17:44.883 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 0, 2024-04-25T12:17:44.883 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 20, 2024-04-25T12:17:44.884 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-25T12:17:44.884 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-25T12:17:44.884 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-25T12:17:44.884 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-25T12:17:44.884 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-25T12:17:44.884 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-25T12:17:44.884 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T12:17:44.884 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-25T12:17:44.884 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-25T12:17:44.884 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-04-25T12:17:44.885 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-25T12:17:44.885 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-25T12:17:44.885 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-25T12:17:44.885 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-25T12:17:44.885 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-25T12:17:44.885 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-25T12:17:44.885 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-25T12:17:44.885 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-25T12:17:44.885 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-25T12:17:44.885 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-25T12:17:44.886 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-25T12:17:44.886 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-25T12:17:44.886 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-25T12:17:44.886 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-25T12:17:44.886 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-25T12:17:44.886 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-25T12:17:44.886 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 96, 2024-04-25T12:17:44.886 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-25T12:17:44.887 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 11, 2024-04-25T12:17:44.887 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 8, 2024-04-25T12:17:44.887 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 8, 2024-04-25T12:17:44.887 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-25T12:17:44.887 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-25T12:17:44.887 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 17, 2024-04-25T12:17:44.887 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 0, 2024-04-25T12:17:44.887 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 2 2024-04-25T12:17:44.887 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.888 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-25T12:17:44.888 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 1, 2024-04-25T12:17:44.888 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-25T12:17:44.888 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-25T12:17:44.888 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-25T12:17:44.888 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.888 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-25T12:17:44.888 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-04-25T12:17:44.888 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.888 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-25T12:17:44.889 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-25T12:17:44.889 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T12:17:44.889 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-25T12:17:44.889 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-04-25T12:17:44.889 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-04-25T12:17:44.889 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-04-25T12:17:44.889 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-25T12:17:44.889 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.889 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T12:17:44.890 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:44.890 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T12:17:44.890 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:44.890 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 802, 2024-04-25T12:17:44.890 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1737749, 2024-04-25T12:17:44.890 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:44.890 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 802, 2024-04-25T12:17:44.890 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T12:17:44.890 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T12:17:44.891 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 802, 2024-04-25T12:17:44.891 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1737749, 2024-04-25T12:17:44.891 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:44.891 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.891 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.891 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.891 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:44.891 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.891 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-25T12:17:44.892 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:44.892 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T12:17:44.892 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:44.892 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T12:17:44.892 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T12:17:44.892 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:44.892 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:44.892 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 44, 2024-04-25T12:17:44.892 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 12407, 2024-04-25T12:17:44.893 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 44, 2024-04-25T12:17:44.893 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 12407, 2024-04-25T12:17:44.893 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:44.893 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.893 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.893 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.893 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:44.893 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.893 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-25T12:17:44.893 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:44.894 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-25T12:17:44.894 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:44.894 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T12:17:44.894 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T12:17:44.894 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:44.894 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:44.894 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 44, 2024-04-25T12:17:44.894 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 44, 2024-04-25T12:17:44.894 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 44, 2024-04-25T12:17:44.895 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 44, 2024-04-25T12:17:44.895 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:44.895 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.895 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.895 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.895 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:44.895 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.895 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-25T12:17:44.896 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:44.896 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T12:17:44.896 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:44.896 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T12:17:44.896 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T12:17:44.896 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:44.896 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:44.896 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T12:17:44.896 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T12:17:44.897 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-25T12:17:44.897 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-25T12:17:44.897 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:44.897 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.897 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.897 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.897 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:44.897 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:44.897 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x55dfeaca8500": { 2024-04-25T12:17:44.897 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:44.898 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T12:17:44.898 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:44.898 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 7, 2024-04-25T12:17:44.898 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 1960, 2024-04-25T12:17:44.898 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:44.898 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 7, 2024-04-25T12:17:44.898 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T12:17:44.898 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T12:17:44.898 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 6, 2024-04-25T12:17:44.899 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1960, 2024-04-25T12:17:44.899 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:44.899 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:44.899 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:44.899 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:44.899 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:44.899 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:44.899 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T12:17:45.551 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T12:17:45.551 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T12:17:45.551 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T12:17:45.551 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=6 2024-04-25T12:17:45.551 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-25T12:17:45.552 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T12:17:45.552 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T12:17:45.552 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T12:17:45.552 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T12:17:45.791 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:17:46.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:46 smithi019 ceph-mon[29213]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:46.640 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:46 smithi169 ceph-mon[32710]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:46.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:46 smithi150 ceph-mon[32250]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:47.548 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T12:17:47.548 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T12:17:47.548 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1440, 2024-04-25T12:17:47.548 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1164, 2024-04-25T12:17:47.549 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 2120430, 2024-04-25T12:17:47.549 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 713320, 2024-04-25T12:17:47.549 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 10, 2024-04-25T12:17:47.549 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 5, 2024-04-25T12:17:47.549 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.178091177, 2024-04-25T12:17:47.549 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.058973912, 2024-04-25T12:17:47.549 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.084291997, 2024-04-25T12:17:47.549 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.040938598, 2024-04-25T12:17:47.550 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:47.550 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1167, 2024-04-25T12:17:47.550 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.385837117, 2024-04-25T12:17:47.550 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000330623 2024-04-25T12:17:47.550 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.550 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:47.550 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1599, 2024-04-25T12:17:47.550 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000357620, 2024-04-25T12:17:47.550 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000223 2024-04-25T12:17:47.551 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.551 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 1664528, 2024-04-25T12:17:47.551 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 458144 2024-04-25T12:17:47.551 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.551 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T12:17:47.551 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 2253, 2024-04-25T12:17:47.551 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 767, 2024-04-25T12:17:47.551 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 835556, 2024-04-25T12:17:47.552 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 462044, 2024-04-25T12:17:47.552 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 18, 2024-04-25T12:17:47.552 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 9, 2024-04-25T12:17:47.552 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.273169406, 2024-04-25T12:17:47.552 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.062926173, 2024-04-25T12:17:47.552 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.100756828, 2024-04-25T12:17:47.552 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.104029630, 2024-04-25T12:17:47.552 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:47.552 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 764, 2024-04-25T12:17:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.442436903, 2024-04-25T12:17:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000579105 2024-04-25T12:17:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2568, 2024-04-25T12:17:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000555046, 2024-04-25T12:17:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000216 2024-04-25T12:17:47.553 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 701296, 2024-04-25T12:17:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 260448 2024-04-25T12:17:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T12:17:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 887, 2024-04-25T12:17:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 618, 2024-04-25T12:17:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 236372, 2024-04-25T12:17:47.554 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 292175, 2024-04-25T12:17:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 12, 2024-04-25T12:17:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 6, 2024-04-25T12:17:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.092848211, 2024-04-25T12:17:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.031266377, 2024-04-25T12:17:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.033851778, 2024-04-25T12:17:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.025175018, 2024-04-25T12:17:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 613, 2024-04-25T12:17:47.555 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.214835094, 2024-04-25T12:17:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000350465 2024-04-25T12:17:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 895, 2024-04-25T12:17:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000214827, 2024-04-25T12:17:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000240 2024-04-25T12:17:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 42528, 2024-04-25T12:17:47.556 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 1136 2024-04-25T12:17:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T12:17:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-25T12:17:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T12:17:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-25T12:17:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T12:17:47.557 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T12:17:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 209, 2024-04-25T12:17:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.036650012, 2024-04-25T12:17:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000175358 2024-04-25T12:17:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-25T12:17:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T12:17:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T12:17:47.558 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T12:17:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000006966, 2024-04-25T12:17:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000003483 2024-04-25T12:17:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-25T12:17:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 143, 2024-04-25T12:17:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 143, 2024-04-25T12:17:47.559 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-25T12:17:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 143, 2024-04-25T12:17:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.121392582, 2024-04-25T12:17:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000848899 2024-04-25T12:17:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-25T12:17:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-04-25T12:17:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-25T12:17:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-25T12:17:47.560 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 12, 2024-04-25T12:17:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-04-25T12:17:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-25T12:17:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 25, 2024-04-25T12:17:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 1, 2024-04-25T12:17:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 24, 2024-04-25T12:17:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-04-25T12:17:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 22, 2024-04-25T12:17:47.561 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-04-25T12:17:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 2, 2024-04-25T12:17:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 2, 2024-04-25T12:17:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 2, 2024-04-25T12:17:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 177, 2024-04-25T12:17:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 169, 2024-04-25T12:17:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-04-25T12:17:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 0, 2024-04-25T12:17:47.562 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-25T12:17:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-25T12:17:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-25T12:17:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-04-25T12:17:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-25T12:17:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-04-25T12:17:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-04-25T12:17:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-04-25T12:17:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-04-25T12:17:47.563 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-25T12:17:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-25T12:17:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-25T12:17:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 4, 2024-04-25T12:17:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 234, 2024-04-25T12:17:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 6, 2024-04-25T12:17:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T12:17:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-25T12:17:47.564 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T12:17:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-25T12:17:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-25T12:17:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T12:17:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T12:17:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-25T12:17:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-04-25T12:17:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 8, 2024-04-25T12:17:47.565 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 5, 2024-04-25T12:17:47.566 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T12:17:47.566 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T12:17:47.566 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T12:17:47.566 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 14, 2024-04-25T12:17:47.566 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 2, 2024-04-25T12:17:47.566 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-04-25T12:17:47.566 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 16 2024-04-25T12:17:47.566 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.566 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-25T12:17:47.567 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 8, 2024-04-25T12:17:47.567 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 0, 2024-04-25T12:17:47.567 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 0, 2024-04-25T12:17:47.567 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 0, 2024-04-25T12:17:47.567 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 20, 2024-04-25T12:17:47.567 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-04-25T12:17:47.567 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-25T12:17:47.567 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-25T12:17:47.568 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-04-25T12:17:47.568 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-04-25T12:17:47.568 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-25T12:17:47.568 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-25T12:17:47.568 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-25T12:17:47.568 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-25T12:17:47.568 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-25T12:17:47.568 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-25T12:17:47.569 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-25T12:17:47.569 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-04-25T12:17:47.569 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-04-25T12:17:47.569 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T12:17:47.569 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-04-25T12:17:47.569 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-25T12:17:47.569 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-04-25T12:17:47.569 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-25T12:17:47.569 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-25T12:17:47.570 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.570 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-25T12:17:47.570 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-25T12:17:47.570 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 100, 2024-04-25T12:17:47.570 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-04-25T12:17:47.570 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-04-25T12:17:47.570 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 100, 2024-04-25T12:17:47.570 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-25T12:17:47.571 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-04-25T12:17:47.571 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 2, 2024-04-25T12:17:47.571 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-04-25T12:17:47.571 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-04-25T12:17:47.571 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 2, 2024-04-25T12:17:47.571 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-04-25T12:17:47.571 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-25T12:17:47.571 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-04-25T12:17:47.571 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-04-25T12:17:47.572 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 4283934, 2024-04-25T12:17:47.572 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 0, 2024-04-25T12:17:47.572 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-25T12:17:47.572 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 81, 2024-04-25T12:17:47.572 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.150794729, 2024-04-25T12:17:47.572 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001861663 2024-04-25T12:17:47.572 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.572 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 0 2024-04-25T12:17:47.573 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.573 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-25T12:17:47.573 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 23, 2024-04-25T12:17:47.573 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 23, 2024-04-25T12:17:47.573 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 0, 2024-04-25T12:17:47.573 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 18, 2024-04-25T12:17:47.573 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 18, 2024-04-25T12:17:47.573 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 0, 2024-04-25T12:17:47.573 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 25, 2024-04-25T12:17:47.574 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 25, 2024-04-25T12:17:47.574 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 0, 2024-04-25T12:17:47.574 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 2, 2024-04-25T12:17:47.574 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 13, 2024-04-25T12:17:47.574 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 11, 2024-04-25T12:17:47.574 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 55276, 2024-04-25T12:17:47.574 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 231800 2024-04-25T12:17:47.574 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.575 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-25T12:17:47.575 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 181, 2024-04-25T12:17:47.575 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 1, 2024-04-25T12:17:47.575 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 143, 2024-04-25T12:17:47.575 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 31, 2024-04-25T12:17:47.575 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 2, 2024-04-25T12:17:47.575 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-25T12:17:47.575 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 4, 2024-04-25T12:17:47.576 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.003207725, 2024-04-25T12:17:47.576 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000801931 2024-04-25T12:17:47.576 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.576 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-25T12:17:47.576 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 97, 2024-04-25T12:17:47.576 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.019278301, 2024-04-25T12:17:47.576 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000198745 2024-04-25T12:17:47.576 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.576 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-25T12:17:47.577 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.577 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.577 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.577 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.577 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-25T12:17:47.577 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.577 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.578 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.578 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.578 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-25T12:17:47.578 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-04-25T12:17:47.578 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000208489, 2024-04-25T12:17:47.578 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000208489 2024-04-25T12:17:47.578 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.578 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-25T12:17:47.578 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.579 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.579 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.579 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.579 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-25T12:17:47.579 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.579 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.579 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.579 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.580 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-25T12:17:47.580 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.580 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.580 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.580 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.580 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-25T12:17:47.580 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.580 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.581 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.581 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.581 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-25T12:17:47.581 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.581 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.581 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.581 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.581 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-25T12:17:47.581 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 10, 2024-04-25T12:17:47.582 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.001587029, 2024-04-25T12:17:47.582 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000158702 2024-04-25T12:17:47.582 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.583 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-25T12:17:47.583 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 6, 2024-04-25T12:17:47.583 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.006867534, 2024-04-25T12:17:47.583 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001144589 2024-04-25T12:17:47.583 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.583 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-25T12:17:47.583 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.584 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.584 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.584 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.584 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-25T12:17:47.584 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 10, 2024-04-25T12:17:47.584 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.051640358, 2024-04-25T12:17:47.584 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.005164035 2024-04-25T12:17:47.584 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.585 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-25T12:17:47.585 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-04-25T12:17:47.585 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000222752, 2024-04-25T12:17:47.585 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000222752 2024-04-25T12:17:47.585 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.585 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-25T12:17:47.585 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.585 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.586 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.586 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.586 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-25T12:17:47.586 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2, 2024-04-25T12:17:47.586 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.002134374, 2024-04-25T12:17:47.586 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.001067187 2024-04-25T12:17:47.586 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.586 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-25T12:17:47.587 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.587 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.587 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.587 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.587 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-25T12:17:47.587 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.587 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.587 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.587 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.588 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-25T12:17:47.588 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.588 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.588 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.588 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.588 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-25T12:17:47.588 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 4, 2024-04-25T12:17:47.588 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.017681495, 2024-04-25T12:17:47.589 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.004420373 2024-04-25T12:17:47.589 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.589 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-25T12:17:47.589 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.589 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.589 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.589 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.589 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-25T12:17:47.590 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1, 2024-04-25T12:17:47.590 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000860028, 2024-04-25T12:17:47.590 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000860028 2024-04-25T12:17:47.590 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.590 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-25T12:17:47.590 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.590 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.590 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.590 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.591 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-25T12:17:47.591 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.591 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.591 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.591 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.591 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-25T12:17:47.591 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.591 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.592 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.592 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.592 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-25T12:17:47.592 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 7, 2024-04-25T12:17:47.592 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.017704497, 2024-04-25T12:17:47.592 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.002529213 2024-04-25T12:17:47.592 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.592 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-25T12:17:47.592 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.593 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.593 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.593 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.593 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-25T12:17:47.593 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.593 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.593 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.593 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.594 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-25T12:17:47.594 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-25T12:17:47.594 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-25T12:17:47.594 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.594 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.594 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.594 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:47.594 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.595 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-25T12:17:47.595 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 2, 2024-04-25T12:17:47.595 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 5, 2024-04-25T12:17:47.595 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 3, 2024-04-25T12:17:47.595 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 2, 2024-04-25T12:17:47.595 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-25T12:17:47.595 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 27, 2024-04-25T12:17:47.595 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 13, 2024-04-25T12:17:47.595 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 610, 2024-04-25T12:17:47.596 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-25T12:17:47.596 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.596 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T12:17:47.596 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T12:17:47.596 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T12:17:47.596 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T12:17:47.596 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T12:17:47.596 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T12:17:47.596 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T12:17:47.597 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T12:17:47.597 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T12:17:47.597 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T12:17:47.597 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T12:17:47.597 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T12:17:47.597 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T12:17:47.597 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T12:17:47.597 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T12:17:47.598 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T12:17:47.598 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T12:17:47.598 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T12:17:47.598 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T12:17:47.598 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T12:17:47.598 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T12:17:47.598 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T12:17:47.598 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T12:17:47.598 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T12:17:47.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T12:17:47.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T12:17:47.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T12:17:47.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T12:17:47.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T12:17:47.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T12:17:47.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T12:17:47.599 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T12:17:47.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T12:17:47.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T12:17:47.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T12:17:47.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T12:17:47.600 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T12:17:47.600 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 375529, 2024-04-25T12:17:47.600 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 158, 2024-04-25T12:17:47.600 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 880, 2024-04-25T12:17:47.600 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 10, 2024-04-25T12:17:47.601 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T12:17:47.601 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T12:17:47.601 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T12:17:47.601 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T12:17:47.601 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T12:17:47.601 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T12:17:47.601 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 8224, 2024-04-25T12:17:47.601 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 171, 2024-04-25T12:17:47.602 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T12:17:47.602 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T12:17:47.602 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T12:17:47.602 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T12:17:47.602 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 96122, 2024-04-25T12:17:47.602 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 605, 2024-04-25T12:17:47.602 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T12:17:47.602 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T12:17:47.603 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T12:17:47.603 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T12:17:47.603 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.603 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-25T12:17:47.603 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-25T12:17:47.603 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-25T12:17:47.603 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 640, 2024-04-25T12:17:47.603 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 103673, 2024-04-25T12:17:47.603 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 37, 2024-04-25T12:17:47.604 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 640, 2024-04-25T12:17:47.604 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-25T12:17:47.604 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 640, 2024-04-25T12:17:47.604 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 20.400569135, 2024-04-25T12:17:47.604 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.031875889 2024-04-25T12:17:47.604 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.604 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-25T12:17:47.604 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-25T12:17:47.605 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 640, 2024-04-25T12:17:47.605 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 671 2024-04-25T12:17:47.605 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.605 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 640, 2024-04-25T12:17:47.605 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 0, 2024-04-25T12:17:47.605 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 640, 2024-04-25T12:17:47.605 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-25T12:17:47.605 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-25T12:17:47.605 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 0, 2024-04-25T12:17:47.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 11, 2024-04-25T12:17:47.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 0, 2024-04-25T12:17:47.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 81, 2024-04-25T12:17:47.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 21, 2024-04-25T12:17:47.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-25T12:17:47.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-25T12:17:47.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-25T12:17:47.606 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-04-25T12:17:47.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 506, 2024-04-25T12:17:47.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-25T12:17:47.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-25T12:17:47.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-25T12:17:47.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 0, 2024-04-25T12:17:47.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 22, 2024-04-25T12:17:47.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-25T12:17:47.607 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-25T12:17:47.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-25T12:17:47.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-25T12:17:47.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-25T12:17:47.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-25T12:17:47.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T12:17:47.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-25T12:17:47.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-25T12:17:47.608 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-04-25T12:17:47.608 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-25T12:17:47.609 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-25T12:17:47.609 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-25T12:17:47.609 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-25T12:17:47.609 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-25T12:17:47.609 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-25T12:17:47.609 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-25T12:17:47.609 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-25T12:17:47.609 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-25T12:17:47.610 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-25T12:17:47.610 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-25T12:17:47.610 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-25T12:17:47.610 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-25T12:17:47.610 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-25T12:17:47.610 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-25T12:17:47.610 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-25T12:17:47.610 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 96, 2024-04-25T12:17:47.611 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-25T12:17:47.611 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 12, 2024-04-25T12:17:47.611 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 12, 2024-04-25T12:17:47.611 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 12, 2024-04-25T12:17:47.611 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-25T12:17:47.611 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-25T12:17:47.611 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 26, 2024-04-25T12:17:47.611 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 0, 2024-04-25T12:17:47.611 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 4 2024-04-25T12:17:47.612 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.612 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-25T12:17:47.612 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 1, 2024-04-25T12:17:47.612 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-25T12:17:47.612 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-25T12:17:47.612 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-25T12:17:47.612 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.612 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-25T12:17:47.613 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-04-25T12:17:47.613 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.613 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-25T12:17:47.613 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-25T12:17:47.613 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T12:17:47.613 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-25T12:17:47.613 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-04-25T12:17:47.613 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-04-25T12:17:47.613 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-04-25T12:17:47.614 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-25T12:17:47.614 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.614 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T12:17:47.614 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:47.614 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T12:17:47.614 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:47.614 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 4576, 2024-04-25T12:17:47.614 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 2686484, 2024-04-25T12:17:47.615 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:47.615 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 4576, 2024-04-25T12:17:47.615 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T12:17:47.615 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T12:17:47.615 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 4576, 2024-04-25T12:17:47.615 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 2686484, 2024-04-25T12:17:47.615 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:47.615 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.615 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.616 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.616 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:47.616 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.616 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-25T12:17:47.616 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:47.616 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T12:17:47.616 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:47.616 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T12:17:47.617 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T12:17:47.617 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:47.617 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:47.617 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 640, 2024-04-25T12:17:47.617 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 103673, 2024-04-25T12:17:47.617 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 640, 2024-04-25T12:17:47.617 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 103673, 2024-04-25T12:17:47.617 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:47.617 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.618 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.618 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.618 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:47.618 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.618 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-25T12:17:47.618 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:47.618 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-25T12:17:47.618 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:47.619 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T12:17:47.619 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T12:17:47.619 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:47.619 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:47.619 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 640, 2024-04-25T12:17:47.619 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 640, 2024-04-25T12:17:47.619 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 640, 2024-04-25T12:17:47.619 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 640, 2024-04-25T12:17:47.619 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:47.620 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.620 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.620 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.620 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:47.620 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.620 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-25T12:17:47.620 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:47.620 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T12:17:47.621 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:47.621 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T12:17:47.621 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T12:17:47.621 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:47.621 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:47.621 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T12:17:47.621 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T12:17:47.621 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-25T12:17:47.621 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-25T12:17:47.622 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:47.622 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.622 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.622 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.622 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:47.622 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:47.622 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x558d79398500": { 2024-04-25T12:17:47.622 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:47.623 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T12:17:47.623 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:47.623 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 100, 2024-04-25T12:17:47.623 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 89630, 2024-04-25T12:17:47.623 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:47.623 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 100, 2024-04-25T12:17:47.623 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T12:17:47.623 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T12:17:47.623 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 81, 2024-04-25T12:17:47.624 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 89630, 2024-04-25T12:17:47.624 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:47.624 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:47.624 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:47.624 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:47.624 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:47.624 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:47.624 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T12:17:48.364 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T12:17:48.365 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T12:17:48.365 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=8 2024-04-25T12:17:48.365 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-04-25T12:17:48.365 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-25T12:17:48.365 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=10 2024-04-25T12:17:48.365 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=4 2024-04-25T12:17:48.365 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T12:17:48.366 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid be62e290-02fa-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T12:17:48.388 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:48 smithi169 ceph-mon[32710]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:48.550 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:48 smithi019 ceph-mon[29213]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:48.603 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T12:17:48.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:48 smithi150 ceph-mon[32250]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:50.361 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:50 smithi169 ceph-mon[32710]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:50.395 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-04-25T12:17:50.396 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T12:17:50.396 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 249, 2024-04-25T12:17:50.396 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 234, 2024-04-25T12:17:50.396 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 1639298, 2024-04-25T12:17:50.396 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 136276, 2024-04-25T12:17:50.396 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 5, 2024-04-25T12:17:50.396 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 1, 2024-04-25T12:17:50.397 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.050543119, 2024-04-25T12:17:50.397 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.015009553, 2024-04-25T12:17:50.397 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.039090134, 2024-04-25T12:17:50.397 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.005484820, 2024-04-25T12:17:50.397 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:50.397 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 233, 2024-04-25T12:17:50.397 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.011828124, 2024-04-25T12:17:50.397 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000050764 2024-04-25T12:17:50.397 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.398 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:50.398 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.398 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.398 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.398 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.398 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 1638464, 2024-04-25T12:17:50.398 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 135952 2024-04-25T12:17:50.398 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.399 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T12:17:50.399 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 688, 2024-04-25T12:17:50.399 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 852, 2024-04-25T12:17:50.399 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 169098, 2024-04-25T12:17:50.399 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 667641, 2024-04-25T12:17:50.399 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 7, 2024-04-25T12:17:50.399 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 5, 2024-04-25T12:17:50.399 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.082037915, 2024-04-25T12:17:50.399 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.024940069, 2024-04-25T12:17:50.400 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.019510474, 2024-04-25T12:17:50.400 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.029337877, 2024-04-25T12:17:50.400 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:50.400 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 849, 2024-04-25T12:17:50.400 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.041777270, 2024-04-25T12:17:50.400 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000049207 2024-04-25T12:17:50.400 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.400 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:50.400 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.401 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.401 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.401 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.401 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T12:17:50.401 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 449488 2024-04-25T12:17:50.401 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.401 INFO:teuthology.orchestra.run.smithi169.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T12:17:50.401 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_messages": 1528, 2024-04-25T12:17:50.401 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages": 1528, 2024-04-25T12:17:50.402 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_bytes": 460101, 2024-04-25T12:17:50.402 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_bytes": 485138, 2024-04-25T12:17:50.402 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_created_connections": 4, 2024-04-25T12:17:50.402 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_active_connections": 3, 2024-04-25T12:17:50.402 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_total_time": 0.161749150, 2024-04-25T12:17:50.402 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_send_time": 0.050427147, 2024-04-25T12:17:50.402 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_recv_time": 0.043059945, 2024-04-25T12:17:50.402 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_running_fast_dispatch_time": 0.054047458, 2024-04-25T12:17:50.402 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T12:17:50.403 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 1527, 2024-04-25T12:17:50.403 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.065642255, 2024-04-25T12:17:50.403 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000042987 2024-04-25T12:17:50.403 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.403 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_handle_ack_lat": { 2024-04-25T12:17:50.403 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.403 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.403 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.403 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.404 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-25T12:17:50.404 INFO:teuthology.orchestra.run.smithi169.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T12:17:50.404 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.404 INFO:teuthology.orchestra.run.smithi169.stdout: "cct": { 2024-04-25T12:17:50.404 INFO:teuthology.orchestra.run.smithi169.stdout: "total_workers": 1, 2024-04-25T12:17:50.404 INFO:teuthology.orchestra.run.smithi169.stdout: "unhealthy_workers": 0 2024-04-25T12:17:50.405 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.405 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-MDSRank": { 2024-04-25T12:17:50.405 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T12:17:50.405 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T12:17:50.405 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 3608, 2024-04-25T12:17:50.405 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.115100296, 2024-04-25T12:17:50.405 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000031901 2024-04-25T12:17:50.405 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:50.405 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.406 INFO:teuthology.orchestra.run.smithi169.stdout: "finisher-PurgeQueue": { 2024-04-25T12:17:50.406 INFO:teuthology.orchestra.run.smithi169.stdout: "queue_len": 0, 2024-04-25T12:17:50.406 INFO:teuthology.orchestra.run.smithi169.stdout: "complete_latency": { 2024-04-25T12:17:50.406 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.406 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.406 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.406 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:50.406 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.407 INFO:teuthology.orchestra.run.smithi169.stdout: "mds": { 2024-04-25T12:17:50.407 INFO:teuthology.orchestra.run.smithi169.stdout: "request": 0, 2024-04-25T12:17:50.407 INFO:teuthology.orchestra.run.smithi169.stdout: "reply": 0, 2024-04-25T12:17:50.407 INFO:teuthology.orchestra.run.smithi169.stdout: "reply_latency": { 2024-04-25T12:17:50.407 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.407 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.407 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.407 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.407 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_reply": 0, 2024-04-25T12:17:50.407 INFO:teuthology.orchestra.run.smithi169.stdout: "forward": 0, 2024-04-25T12:17:50.408 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_complete": 0, 2024-04-25T12:17:50.408 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_fetch_keys": 0, 2024-04-25T12:17:50.408 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_commit": 0, 2024-04-25T12:17:50.408 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_split": 0, 2024-04-25T12:17:50.408 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_merge": 0, 2024-04-25T12:17:50.408 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes": 0, 2024-04-25T12:17:50.408 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_top": 0, 2024-04-25T12:17:50.408 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_bottom": 0, 2024-04-25T12:17:50.408 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pin_tail": 0, 2024-04-25T12:17:50.409 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_pinned": 0, 2024-04-25T12:17:50.409 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_expired": 0, 2024-04-25T12:17:50.409 INFO:teuthology.orchestra.run.smithi169.stdout: "inodes_with_caps": 0, 2024-04-25T12:17:50.409 INFO:teuthology.orchestra.run.smithi169.stdout: "caps": 0, 2024-04-25T12:17:50.409 INFO:teuthology.orchestra.run.smithi169.stdout: "subtrees": 1, 2024-04-25T12:17:50.409 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse": 0, 2024-04-25T12:17:50.409 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_hit": 0, 2024-04-25T12:17:50.409 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_forward": 0, 2024-04-25T12:17:50.410 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_discover": 0, 2024-04-25T12:17:50.410 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_dir_fetch": 0, 2024-04-25T12:17:50.410 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_remote_ino": 0, 2024-04-25T12:17:50.410 INFO:teuthology.orchestra.run.smithi169.stdout: "traverse_lock": 0, 2024-04-25T12:17:50.410 INFO:teuthology.orchestra.run.smithi169.stdout: "load_cent": 0, 2024-04-25T12:17:50.410 INFO:teuthology.orchestra.run.smithi169.stdout: "q": 0, 2024-04-25T12:17:50.410 INFO:teuthology.orchestra.run.smithi169.stdout: "exported": 0, 2024-04-25T12:17:50.410 INFO:teuthology.orchestra.run.smithi169.stdout: "exported_inodes": 0, 2024-04-25T12:17:50.411 INFO:teuthology.orchestra.run.smithi169.stdout: "imported": 0, 2024-04-25T12:17:50.411 INFO:teuthology.orchestra.run.smithi169.stdout: "imported_inodes": 0, 2024-04-25T12:17:50.411 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_dir_fetch": 0, 2024-04-25T12:17:50.411 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_backtrace_fetch": 0, 2024-04-25T12:17:50.411 INFO:teuthology.orchestra.run.smithi169.stdout: "openino_peer_discover": 0, 2024-04-25T12:17:50.411 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rfiles": 0, 2024-04-25T12:17:50.411 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rbytes": 0, 2024-04-25T12:17:50.411 INFO:teuthology.orchestra.run.smithi169.stdout: "root_rsnaps": 0, 2024-04-25T12:17:50.411 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T12:17:50.412 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_set_tag": 0, 2024-04-25T12:17:50.412 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T12:17:50.412 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_inotable_repaired": 0, 2024-04-25T12:17:50.412 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_inodes": 0, 2024-04-25T12:17:50.412 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T12:17:50.412 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T12:17:50.412 INFO:teuthology.orchestra.run.smithi169.stdout: "scrub_file_inodes": 0, 2024-04-25T12:17:50.412 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_inode_file_caps": 0, 2024-04-25T12:17:50.413 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T12:17:50.413 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_grant": 0, 2024-04-25T12:17:50.413 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T12:17:50.413 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T12:17:50.413 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T12:17:50.413 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps": 0, 2024-04-25T12:17:50.413 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_caps_dirty": 0, 2024-04-25T12:17:50.413 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_cap_release": 0, 2024-04-25T12:17:50.414 INFO:teuthology.orchestra.run.smithi169.stdout: "process_request_cap_release": 0 2024-04-25T12:17:50.414 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.414 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_cache": { 2024-04-25T12:17:50.414 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update": 0, 2024-04-25T12:17:50.414 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_update_receipt": 0, 2024-04-25T12:17:50.414 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_try_discover": 0, 2024-04-25T12:17:50.414 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_send_discover": 0, 2024-04-25T12:17:50.414 INFO:teuthology.orchestra.run.smithi169.stdout: "dir_handle_discover": 0, 2024-04-25T12:17:50.414 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays": 0, 2024-04-25T12:17:50.415 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_delayed": 0, 2024-04-25T12:17:50.415 INFO:teuthology.orchestra.run.smithi169.stdout: "num_strays_enqueuing": 0, 2024-04-25T12:17:50.415 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_created": 0, 2024-04-25T12:17:50.415 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_enqueued": 0, 2024-04-25T12:17:50.415 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_reintegrated": 0, 2024-04-25T12:17:50.415 INFO:teuthology.orchestra.run.smithi169.stdout: "strays_migrated": 0, 2024-04-25T12:17:50.415 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_processing": 0, 2024-04-25T12:17:50.415 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_enqueued": 0, 2024-04-25T12:17:50.416 INFO:teuthology.orchestra.run.smithi169.stdout: "num_recovering_prioritized": 0, 2024-04-25T12:17:50.416 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_started": 0, 2024-04-25T12:17:50.416 INFO:teuthology.orchestra.run.smithi169.stdout: "recovery_completed": 0, 2024-04-25T12:17:50.416 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_path": 0, 2024-04-25T12:17:50.416 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_quiesce_inode": 0, 2024-04-25T12:17:50.416 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T12:17:50.416 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_exportdir": 0, 2024-04-25T12:17:50.416 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_flush": 0, 2024-04-25T12:17:50.416 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragmentdir": 0, 2024-04-25T12:17:50.417 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_fragstats": 0, 2024-04-25T12:17:50.417 INFO:teuthology.orchestra.run.smithi169.stdout: "ireq_inodestats": 0 2024-04-25T12:17:50.417 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.417 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_log": { 2024-04-25T12:17:50.417 INFO:teuthology.orchestra.run.smithi169.stdout: "evlrg": 0, 2024-04-25T12:17:50.417 INFO:teuthology.orchestra.run.smithi169.stdout: "evadd": 0, 2024-04-25T12:17:50.417 INFO:teuthology.orchestra.run.smithi169.stdout: "evex": 0, 2024-04-25T12:17:50.417 INFO:teuthology.orchestra.run.smithi169.stdout: "evtrm": 0, 2024-04-25T12:17:50.417 INFO:teuthology.orchestra.run.smithi169.stdout: "ev": 6, 2024-04-25T12:17:50.418 INFO:teuthology.orchestra.run.smithi169.stdout: "evexg": 0, 2024-04-25T12:17:50.418 INFO:teuthology.orchestra.run.smithi169.stdout: "evexd": 0, 2024-04-25T12:17:50.418 INFO:teuthology.orchestra.run.smithi169.stdout: "segadd": 0, 2024-04-25T12:17:50.418 INFO:teuthology.orchestra.run.smithi169.stdout: "segex": 0, 2024-04-25T12:17:50.418 INFO:teuthology.orchestra.run.smithi169.stdout: "segtrm": 0, 2024-04-25T12:17:50.418 INFO:teuthology.orchestra.run.smithi169.stdout: "seg": 2, 2024-04-25T12:17:50.418 INFO:teuthology.orchestra.run.smithi169.stdout: "segmjr": 2, 2024-04-25T12:17:50.418 INFO:teuthology.orchestra.run.smithi169.stdout: "segexg": 0, 2024-04-25T12:17:50.419 INFO:teuthology.orchestra.run.smithi169.stdout: "segexd": 0, 2024-04-25T12:17:50.419 INFO:teuthology.orchestra.run.smithi169.stdout: "expos": 4194304, 2024-04-25T12:17:50.419 INFO:teuthology.orchestra.run.smithi169.stdout: "wrpos": 4195956, 2024-04-25T12:17:50.419 INFO:teuthology.orchestra.run.smithi169.stdout: "rdpos": 4195815, 2024-04-25T12:17:50.419 INFO:teuthology.orchestra.run.smithi169.stdout: "jlat": { 2024-04-25T12:17:50.419 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.419 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.419 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.419 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.420 INFO:teuthology.orchestra.run.smithi169.stdout: "replayed": 6 2024-04-25T12:17:50.420 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.420 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_mem": { 2024-04-25T12:17:50.420 INFO:teuthology.orchestra.run.smithi169.stdout: "ino": 3, 2024-04-25T12:17:50.420 INFO:teuthology.orchestra.run.smithi169.stdout: "ino+": 3, 2024-04-25T12:17:50.420 INFO:teuthology.orchestra.run.smithi169.stdout: "ino-": 0, 2024-04-25T12:17:50.420 INFO:teuthology.orchestra.run.smithi169.stdout: "dir": 1, 2024-04-25T12:17:50.420 INFO:teuthology.orchestra.run.smithi169.stdout: "dir+": 1, 2024-04-25T12:17:50.420 INFO:teuthology.orchestra.run.smithi169.stdout: "dir-": 0, 2024-04-25T12:17:50.421 INFO:teuthology.orchestra.run.smithi169.stdout: "dn": 0, 2024-04-25T12:17:50.421 INFO:teuthology.orchestra.run.smithi169.stdout: "dn+": 0, 2024-04-25T12:17:50.421 INFO:teuthology.orchestra.run.smithi169.stdout: "dn-": 0, 2024-04-25T12:17:50.421 INFO:teuthology.orchestra.run.smithi169.stdout: "cap": 0, 2024-04-25T12:17:50.421 INFO:teuthology.orchestra.run.smithi169.stdout: "cap+": 0, 2024-04-25T12:17:50.421 INFO:teuthology.orchestra.run.smithi169.stdout: "cap-": 0, 2024-04-25T12:17:50.421 INFO:teuthology.orchestra.run.smithi169.stdout: "rss": 50388, 2024-04-25T12:17:50.421 INFO:teuthology.orchestra.run.smithi169.stdout: "heap": 215416 2024-04-25T12:17:50.422 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.422 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_server": { 2024-04-25T12:17:50.422 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_client_request": 0, 2024-04-25T12:17:50.422 INFO:teuthology.orchestra.run.smithi169.stdout: "dispatch_server_request": 0, 2024-04-25T12:17:50.422 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_request": 0, 2024-04-25T12:17:50.422 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_client_session": 0, 2024-04-25T12:17:50.422 INFO:teuthology.orchestra.run.smithi169.stdout: "handle_peer_request": 0, 2024-04-25T12:17:50.422 INFO:teuthology.orchestra.run.smithi169.stdout: "req_create_latency": { 2024-04-25T12:17:50.423 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.423 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.423 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.423 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.423 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getattr_latency": { 2024-04-25T12:17:50.423 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.423 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.423 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.423 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.423 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getfilelock_latency": { 2024-04-25T12:17:50.424 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.424 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.424 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.424 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.424 INFO:teuthology.orchestra.run.smithi169.stdout: "req_link_latency": { 2024-04-25T12:17:50.424 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.424 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.424 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.425 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.425 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookup_latency": { 2024-04-25T12:17:50.425 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.425 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.425 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.425 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.425 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookuphash_latency": { 2024-04-25T12:17:50.425 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.426 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.426 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.426 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.426 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupino_latency": { 2024-04-25T12:17:50.426 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.426 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.426 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.426 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.426 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupname_latency": { 2024-04-25T12:17:50.427 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.427 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.427 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.427 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.427 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupparent_latency": { 2024-04-25T12:17:50.427 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.427 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.427 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.427 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.428 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lookupsnap_latency": { 2024-04-25T12:17:50.428 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.428 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.428 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.428 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.428 INFO:teuthology.orchestra.run.smithi169.stdout: "req_lssnap_latency": { 2024-04-25T12:17:50.428 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.428 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.429 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.429 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.429 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mkdir_latency": { 2024-04-25T12:17:50.429 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mknod_latency": { 2024-04-25T12:17:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.430 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: "req_mksnap_latency": { 2024-04-25T12:17:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: "req_open_latency": { 2024-04-25T12:17:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.431 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.432 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.432 INFO:teuthology.orchestra.run.smithi169.stdout: "req_readdir_latency": { 2024-04-25T12:17:50.432 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.432 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.432 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.432 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.432 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rename_latency": { 2024-04-25T12:17:50.432 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.432 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.433 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.433 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.433 INFO:teuthology.orchestra.run.smithi169.stdout: "req_renamesnap_latency": { 2024-04-25T12:17:50.433 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.433 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.433 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.433 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.433 INFO:teuthology.orchestra.run.smithi169.stdout: "req_snapdiff_latency": { 2024-04-25T12:17:50.434 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.434 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.434 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.434 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.434 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmdir_latency": { 2024-04-25T12:17:50.434 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.434 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.434 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.434 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmsnap_latency": { 2024-04-25T12:17:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: "req_rmxattr_latency": { 2024-04-25T12:17:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.435 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setattr_latency": { 2024-04-25T12:17:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setdirlayout_latency": { 2024-04-25T12:17:50.436 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.437 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.437 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.437 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.437 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setfilelock_latency": { 2024-04-25T12:17:50.437 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.437 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.437 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.437 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.437 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setlayout_latency": { 2024-04-25T12:17:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: "req_setxattr_latency": { 2024-04-25T12:17:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.438 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: "req_symlink_latency": { 2024-04-25T12:17:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: "req_unlink_latency": { 2024-04-25T12:17:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.439 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_revoke_eviction": 0, 2024-04-25T12:17:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: "cap_acquisition_throttle": 0, 2024-04-25T12:17:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: "req_getvxattr_latency": { 2024-04-25T12:17:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.440 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_sessions": { 2024-04-25T12:17:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: "session_count": 0, 2024-04-25T12:17:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: "session_add": 2, 2024-04-25T12:17:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: "session_remove": 2, 2024-04-25T12:17:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_open": 1, 2024-04-25T12:17:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: "sessions_stale": 0, 2024-04-25T12:17:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: "total_load": 0, 2024-04-25T12:17:50.441 INFO:teuthology.orchestra.run.smithi169.stdout: "average_load": 0, 2024-04-25T12:17:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: "avg_session_uptime": 1196, 2024-04-25T12:17:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: "mdthresh_evicted": 0 2024-04-25T12:17:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: "mempool": { 2024-04-25T12:17:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_bytes": 0, 2024-04-25T12:17:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: "bloom_filter_items": 0, 2024-04-25T12:17:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T12:17:50.442 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_alloc_items": 0, 2024-04-25T12:17:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T12:17:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_data_items": 0, 2024-04-25T12:17:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T12:17:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T12:17:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T12:17:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T12:17:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T12:17:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_other_items": 0, 2024-04-25T12:17:50.443 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T12:17:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T12:17:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_bytes": 0, 2024-04-25T12:17:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_extent_items": 0, 2024-04-25T12:17:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_bytes": 0, 2024-04-25T12:17:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_blob_items": 0, 2024-04-25T12:17:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T12:17:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T12:17:50.444 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T12:17:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T12:17:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T12:17:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_fsck_items": 0, 2024-04-25T12:17:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_bytes": 0, 2024-04-25T12:17:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_txc_items": 0, 2024-04-25T12:17:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T12:17:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T12:17:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_bytes": 0, 2024-04-25T12:17:50.445 INFO:teuthology.orchestra.run.smithi169.stdout: "bluestore_writing_items": 0, 2024-04-25T12:17:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_bytes": 0, 2024-04-25T12:17:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_items": 0, 2024-04-25T12:17:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T12:17:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_reader_items": 0, 2024-04-25T12:17:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T12:17:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: "bluefs_file_writer_items": 0, 2024-04-25T12:17:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_bytes": 189393, 2024-04-25T12:17:50.446 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_anon_items": 51, 2024-04-25T12:17:50.447 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_bytes": 0, 2024-04-25T12:17:50.447 INFO:teuthology.orchestra.run.smithi169.stdout: "buffer_meta_items": 0, 2024-04-25T12:17:50.447 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_bytes": 0, 2024-04-25T12:17:50.447 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_items": 0, 2024-04-25T12:17:50.447 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_bytes": 0, 2024-04-25T12:17:50.447 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_mapbl_items": 0, 2024-04-25T12:17:50.447 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_bytes": 0, 2024-04-25T12:17:50.447 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_pglog_items": 0, 2024-04-25T12:17:50.448 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_bytes": 8224, 2024-04-25T12:17:50.448 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_items": 171, 2024-04-25T12:17:50.448 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T12:17:50.448 INFO:teuthology.orchestra.run.smithi169.stdout: "osdmap_mapping_items": 0, 2024-04-25T12:17:50.448 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_bytes": 0, 2024-04-25T12:17:50.448 INFO:teuthology.orchestra.run.smithi169.stdout: "pgmap_items": 0, 2024-04-25T12:17:50.448 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_bytes": 8512, 2024-04-25T12:17:50.448 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_co_items": 13, 2024-04-25T12:17:50.449 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_bytes": 0, 2024-04-25T12:17:50.449 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_1_items": 0, 2024-04-25T12:17:50.449 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_bytes": 0, 2024-04-25T12:17:50.449 INFO:teuthology.orchestra.run.smithi169.stdout: "unittest_2_items": 0 2024-04-25T12:17:50.449 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.449 INFO:teuthology.orchestra.run.smithi169.stdout: "objecter": { 2024-04-25T12:17:50.449 INFO:teuthology.orchestra.run.smithi169.stdout: "op_active": 0, 2024-04-25T12:17:50.449 INFO:teuthology.orchestra.run.smithi169.stdout: "op_laggy": 0, 2024-04-25T12:17:50.449 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send": 2211, 2024-04-25T12:17:50.450 INFO:teuthology.orchestra.run.smithi169.stdout: "op_send_bytes": 21, 2024-04-25T12:17:50.450 INFO:teuthology.orchestra.run.smithi169.stdout: "op_resend": 3, 2024-04-25T12:17:50.450 INFO:teuthology.orchestra.run.smithi169.stdout: "op_reply": 2211, 2024-04-25T12:17:50.450 INFO:teuthology.orchestra.run.smithi169.stdout: "op_latency": { 2024-04-25T12:17:50.450 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2211, 2024-04-25T12:17:50.450 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 1.210974099, 2024-04-25T12:17:50.450 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000547704 2024-04-25T12:17:50.450 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.451 INFO:teuthology.orchestra.run.smithi169.stdout: "op_inflight": 0, 2024-04-25T12:17:50.451 INFO:teuthology.orchestra.run.smithi169.stdout: "oplen_avg": { 2024-04-25T12:17:50.451 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 2211, 2024-04-25T12:17:50.451 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 2212 2024-04-25T12:17:50.451 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.451 INFO:teuthology.orchestra.run.smithi169.stdout: "op": 2211, 2024-04-25T12:17:50.451 INFO:teuthology.orchestra.run.smithi169.stdout: "op_r": 2211, 2024-04-25T12:17:50.451 INFO:teuthology.orchestra.run.smithi169.stdout: "op_w": 0, 2024-04-25T12:17:50.451 INFO:teuthology.orchestra.run.smithi169.stdout: "op_rmw": 0, 2024-04-25T12:17:50.452 INFO:teuthology.orchestra.run.smithi169.stdout: "op_pg": 0, 2024-04-25T12:17:50.452 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_stat": 1443, 2024-04-25T12:17:50.452 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_create": 0, 2024-04-25T12:17:50.452 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_read": 766, 2024-04-25T12:17:50.452 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_write": 0, 2024-04-25T12:17:50.452 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writefull": 0, 2024-04-25T12:17:50.452 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_writesame": 0, 2024-04-25T12:17:50.452 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_append": 0, 2024-04-25T12:17:50.452 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_zero": 0, 2024-04-25T12:17:50.452 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_truncate": 0, 2024-04-25T12:17:50.453 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_delete": 0, 2024-04-25T12:17:50.453 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_mapext": 0, 2024-04-25T12:17:50.453 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_sparse_read": 0, 2024-04-25T12:17:50.453 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_clonerange": 0, 2024-04-25T12:17:50.453 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_getxattr": 1, 2024-04-25T12:17:50.453 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_setxattr": 0, 2024-04-25T12:17:50.453 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_cmpxattr": 0, 2024-04-25T12:17:50.453 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_rmxattr": 0, 2024-04-25T12:17:50.454 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_resetxattrs": 0, 2024-04-25T12:17:50.454 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_call": 0, 2024-04-25T12:17:50.454 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_watch": 0, 2024-04-25T12:17:50.454 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_notify": 0, 2024-04-25T12:17:50.454 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T12:17:50.454 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls": 0, 2024-04-25T12:17:50.454 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_pgls_filter": 0, 2024-04-25T12:17:50.454 INFO:teuthology.orchestra.run.smithi169.stdout: "osdop_other": 0, 2024-04-25T12:17:50.454 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_active": 0, 2024-04-25T12:17:50.455 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_send": 0, 2024-04-25T12:17:50.455 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_resend": 0, 2024-04-25T12:17:50.455 INFO:teuthology.orchestra.run.smithi169.stdout: "linger_ping": 0, 2024-04-25T12:17:50.455 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_active": 0, 2024-04-25T12:17:50.455 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_send": 0, 2024-04-25T12:17:50.455 INFO:teuthology.orchestra.run.smithi169.stdout: "poolop_resend": 0, 2024-04-25T12:17:50.455 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_active": 0, 2024-04-25T12:17:50.455 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_send": 0, 2024-04-25T12:17:50.455 INFO:teuthology.orchestra.run.smithi169.stdout: "poolstat_resend": 0, 2024-04-25T12:17:50.456 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_active": 0, 2024-04-25T12:17:50.456 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_send": 0, 2024-04-25T12:17:50.456 INFO:teuthology.orchestra.run.smithi169.stdout: "statfs_resend": 0, 2024-04-25T12:17:50.456 INFO:teuthology.orchestra.run.smithi169.stdout: "command_active": 0, 2024-04-25T12:17:50.456 INFO:teuthology.orchestra.run.smithi169.stdout: "command_send": 0, 2024-04-25T12:17:50.456 INFO:teuthology.orchestra.run.smithi169.stdout: "command_resend": 0, 2024-04-25T12:17:50.456 INFO:teuthology.orchestra.run.smithi169.stdout: "map_epoch": 96, 2024-04-25T12:17:50.456 INFO:teuthology.orchestra.run.smithi169.stdout: "map_full": 0, 2024-04-25T12:17:50.457 INFO:teuthology.orchestra.run.smithi169.stdout: "map_inc": 11, 2024-04-25T12:17:50.457 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_sessions": 5, 2024-04-25T12:17:50.457 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_open": 5, 2024-04-25T12:17:50.457 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_session_close": 0, 2024-04-25T12:17:50.457 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_laggy": 0, 2024-04-25T12:17:50.457 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_wr": 0, 2024-04-25T12:17:50.457 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_rd": 2, 2024-04-25T12:17:50.457 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_del": 0 2024-04-25T12:17:50.457 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.458 INFO:teuthology.orchestra.run.smithi169.stdout: "oft": { 2024-04-25T12:17:50.458 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_objs": 0, 2024-04-25T12:17:50.458 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_kv_pairs": 0, 2024-04-25T12:17:50.458 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_updates": 0, 2024-04-25T12:17:50.458 INFO:teuthology.orchestra.run.smithi169.stdout: "omap_total_removes": 0 2024-04-25T12:17:50.458 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.458 INFO:teuthology.orchestra.run.smithi169.stdout: "osd-slow-ops": { 2024-04-25T12:17:50.458 INFO:teuthology.orchestra.run.smithi169.stdout: "slow_ops_count": 0 2024-04-25T12:17:50.459 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.459 INFO:teuthology.orchestra.run.smithi169.stdout: "purge_queue": { 2024-04-25T12:17:50.459 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops": 0, 2024-04-25T12:17:50.459 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T12:17:50.459 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing": 0, 2024-04-25T12:17:50.459 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executing_high_water": 0, 2024-04-25T12:17:50.459 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed_ops": 0, 2024-04-25T12:17:50.459 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_executed": 0, 2024-04-25T12:17:50.460 INFO:teuthology.orchestra.run.smithi169.stdout: "pq_item_in_journal": 0 2024-04-25T12:17:50.460 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.460 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T12:17:50.460 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:50.460 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T12:17:50.460 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:50.460 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 2461, 2024-04-25T12:17:50.460 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 2031330, 2024-04-25T12:17:50.460 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:50.461 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 2461, 2024-04-25T12:17:50.461 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T12:17:50.461 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T12:17:50.461 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2461, 2024-04-25T12:17:50.461 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 2031330, 2024-04-25T12:17:50.461 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:50.461 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.461 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.461 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.462 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:50.462 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.462 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_bytes": { 2024-04-25T12:17:50.462 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:50.462 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 104857600, 2024-04-25T12:17:50.462 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:50.462 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T12:17:50.462 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T12:17:50.463 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:50.463 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:50.463 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 2211, 2024-04-25T12:17:50.463 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 1657, 2024-04-25T12:17:50.463 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2211, 2024-04-25T12:17:50.463 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 1657, 2024-04-25T12:17:50.463 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:50.463 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.463 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.464 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.464 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:50.464 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.464 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-objecter_ops": { 2024-04-25T12:17:50.464 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:50.464 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 1024, 2024-04-25T12:17:50.464 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:50.464 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T12:17:50.465 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T12:17:50.465 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:50.465 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:50.465 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 2211, 2024-04-25T12:17:50.465 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 2211, 2024-04-25T12:17:50.465 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 2211, 2024-04-25T12:17:50.465 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 2211, 2024-04-25T12:17:50.465 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:50.465 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.466 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.466 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.466 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:50.466 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.466 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle": { 2024-04-25T12:17:50.466 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:50.466 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T12:17:50.466 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:50.467 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T12:17:50.467 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T12:17:50.467 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:50.467 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:50.467 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T12:17:50.467 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T12:17:50.467 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-25T12:17:50.467 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-25T12:17:50.467 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:50.468 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.468 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.468 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.468 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:50.468 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-04-25T12:17:50.468 INFO:teuthology.orchestra.run.smithi169.stdout: "throttle-write_buf_throttle-0x563cd72c86e0": { 2024-04-25T12:17:50.468 INFO:teuthology.orchestra.run.smithi169.stdout: "val": 0, 2024-04-25T12:17:50.468 INFO:teuthology.orchestra.run.smithi169.stdout: "max": 3758096384, 2024-04-25T12:17:50.468 INFO:teuthology.orchestra.run.smithi169.stdout: "get_started": 0, 2024-04-25T12:17:50.469 INFO:teuthology.orchestra.run.smithi169.stdout: "get": 0, 2024-04-25T12:17:50.469 INFO:teuthology.orchestra.run.smithi169.stdout: "get_sum": 0, 2024-04-25T12:17:50.469 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_fail": 0, 2024-04-25T12:17:50.469 INFO:teuthology.orchestra.run.smithi169.stdout: "get_or_fail_success": 0, 2024-04-25T12:17:50.469 INFO:teuthology.orchestra.run.smithi169.stdout: "take": 0, 2024-04-25T12:17:50.469 INFO:teuthology.orchestra.run.smithi169.stdout: "take_sum": 0, 2024-04-25T12:17:50.469 INFO:teuthology.orchestra.run.smithi169.stdout: "put": 0, 2024-04-25T12:17:50.469 INFO:teuthology.orchestra.run.smithi169.stdout: "put_sum": 0, 2024-04-25T12:17:50.470 INFO:teuthology.orchestra.run.smithi169.stdout: "wait": { 2024-04-25T12:17:50.470 INFO:teuthology.orchestra.run.smithi169.stdout: "avgcount": 0, 2024-04-25T12:17:50.470 INFO:teuthology.orchestra.run.smithi169.stdout: "sum": 0.000000000, 2024-04-25T12:17:50.470 INFO:teuthology.orchestra.run.smithi169.stdout: "avgtime": 0.000000000 2024-04-25T12:17:50.470 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:50.470 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-04-25T12:17:50.470 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-04-25T12:17:50.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:50 smithi019 ceph-mon[29213]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:50.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:50 smithi150 ceph-mon[32250]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:51.239 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-25T12:17:51.239 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-25T12:17:51.239 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-25T12:17:51.239 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-04-25T12:17:51.239 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-25T12:17:51.239 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-25T12:17:51.240 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T12:17:51.240 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2024-04-25T12:17:51.249 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T12:17:51.260 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T12:17:51.260 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T12:17:51.260 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T12:17:51.286 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T12:17:51.287 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T12:17:51.347 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:17:51.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.504+0000 7f8e4ace1640 1 -- 172.21.15.19:0/4198182961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e44107280 msgr2=0x7f8e4410ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:51.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.504+0000 7f8e4ace1640 1 --2- 172.21.15.19:0/4198182961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e44107280 0x7f8e4410ee50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009ca0 tx=0x7f8e3c02a8c0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.504+0000 7f8e4ace1640 1 -- 172.21.15.19:0/4198182961 shutdown_connections 2024-04-25T12:17:51.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.504+0000 7f8e4ace1640 1 --2- 172.21.15.19:0/4198182961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e44107280 0x7f8e4410ee50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.504+0000 7f8e4ace1640 1 --2- 172.21.15.19:0/4198182961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e44102bf0 0x7f8e44106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.504+0000 7f8e4ace1640 1 --2- 172.21.15.19:0/4198182961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e44102240 0x7f8e44102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.504+0000 7f8e4ace1640 1 -- 172.21.15.19:0/4198182961 >> 172.21.15.19:0/4198182961 conn(0x7f8e440790e0 msgr2=0x7f8e440794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:51.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.504+0000 7f8e4ace1640 1 -- 172.21.15.19:0/4198182961 shutdown_connections 2024-04-25T12:17:51.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.504+0000 7f8e4ace1640 1 -- 172.21.15.19:0/4198182961 wait complete. 2024-04-25T12:17:51.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.504+0000 7f8e4ace1640 1 Processor -- start 2024-04-25T12:17:51.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.504+0000 7f8e4ace1640 1 -- start start 2024-04-25T12:17:51.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e4ace1640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e44102240 0x7f8e4419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:51.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e4ace1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e44107280 0x7f8e4419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:51.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e4ace1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e44198640 0x7f8e44198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:51.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e4ace1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e44112320 con 0x7f8e44107280 2024-04-25T12:17:51.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e4ace1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8e441121a0 con 0x7f8e44198640 2024-04-25T12:17:51.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e4ace1640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f8e441124a0 con 0x7f8e44102240 2024-04-25T12:17:51.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e3bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e44107280 0x7f8e4419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:51.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e3bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e44107280 0x7f8e4419e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33684/0 (socket says 172.21.15.19:33684) 2024-04-25T12:17:51.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e3bfff640 1 -- 172.21.15.19:0/3844894375 learned_addr learned my addr 172.21.15.19:0/3844894375 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:51.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e48a7d640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e44102240 0x7f8e4419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:51.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e4927e640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e44198640 0x7f8e44198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:51.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e3bfff640 1 -- 172.21.15.19:0/3844894375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e44102240 msgr2=0x7f8e4419e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:51.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e3bfff640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e44102240 0x7f8e4419e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e3bfff640 1 -- 172.21.15.19:0/3844894375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e44198640 msgr2=0x7f8e44198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:51.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e3bfff640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e44198640 0x7f8e44198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e3bfff640 1 -- 172.21.15.19:0/3844894375 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e44199160 con 0x7f8e44107280 2024-04-25T12:17:51.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e4927e640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e44198640 0x7f8e44198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:17:51.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e3bfff640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e44107280 0x7f8e4419e980 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400e9f0 tx=0x7f8e3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:51.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e39ffb640 1 -- 172.21.15.19:0/3844894375 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400ccc0 con 0x7f8e44107280 2024-04-25T12:17:51.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e4ace1640 1 -- 172.21.15.19:0/3844894375 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e44199450 con 0x7f8e44107280 2024-04-25T12:17:51.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.505+0000 7f8e39ffb640 1 -- 172.21.15.19:0/3844894375 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f8e34010900 con 0x7f8e44107280 2024-04-25T12:17:51.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.506+0000 7f8e39ffb640 1 -- 172.21.15.19:0/3844894375 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34018900 con 0x7f8e44107280 2024-04-25T12:17:51.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.506+0000 7f8e4ace1640 1 -- 172.21.15.19:0/3844894375 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e441a5650 con 0x7f8e44107280 2024-04-25T12:17:51.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.507+0000 7f8e4ace1640 1 -- 172.21.15.19:0/3844894375 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e440691c0 con 0x7f8e44107280 2024-04-25T12:17:51.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.508+0000 7f8e39ffb640 1 -- 172.21.15.19:0/3844894375 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8e34010ab0 con 0x7f8e44107280 2024-04-25T12:17:51.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.509+0000 7f8e39ffb640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f8e2c0a3d20 0x7f8e2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:51.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.509+0000 7f8e39ffb640 1 -- 172.21.15.19:0/3844894375 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f8e340d0c00 con 0x7f8e44107280 2024-04-25T12:17:51.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.509+0000 7f8e48a7d640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f8e2c0a3d20 0x7f8e2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:51.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.509+0000 7f8e48a7d640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f8e2c0a3d20 0x7f8e2c0a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009b60 tx=0x7f8e40005f90 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:51.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.513+0000 7f8e39ffb640 1 -- 172.21.15.19:0/3844894375 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e3409d210 con 0x7f8e44107280 2024-04-25T12:17:51.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.703+0000 7f8e4ace1640 1 -- 172.21.15.19:0/3844894375 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e4410a480 con 0x7f8e44107280 2024-04-25T12:17:51.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.705+0000 7f8e39ffb640 1 -- 172.21.15.19:0/3844894375 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+23199 (secure 0 0 0) 0x7f8e3409cb10 con 0x7f8e44107280 2024-04-25T12:17:51.706 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:17:51.707 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":96,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:17:15.346984+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:05:01.658354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:05:02.969078+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/2813829603":"2024-04-26T12:07:09.072296+0000","172.21.15.19:0/1048024792":"2024-04-26T12:07:09.072296+0000","172.21.15.19:6800/2853466745":"2024-04-26T12:07:09.072296+0000","172.21.15.19:0/1094943085":"2024-04-26T12:07:09.072296+0000","172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/2734487323":"2024-04-26T12:07:09.072296+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/2853466745":"2024-04-26T12:07:09.072296+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:17:51.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.707+0000 7f8e4ace1640 1 -- 172.21.15.19:0/3844894375 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f8e2c0a3d20 msgr2=0x7f8e2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:51.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.707+0000 7f8e4ace1640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f8e2c0a3d20 0x7f8e2c0a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009b60 tx=0x7f8e40005f90 comp rx=0 tx=0).stop 2024-04-25T12:17:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.707+0000 7f8e4ace1640 1 -- 172.21.15.19:0/3844894375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e44107280 msgr2=0x7f8e4419e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.707+0000 7f8e4ace1640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e44107280 0x7f8e4419e980 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400e9f0 tx=0x7f8e3400eeb0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.707+0000 7f8e4ace1640 1 -- 172.21.15.19:0/3844894375 shutdown_connections 2024-04-25T12:17:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.707+0000 7f8e4ace1640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f8e2c0a3d20 0x7f8e2c0a61e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.707+0000 7f8e4ace1640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e44198640 0x7f8e44198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.707+0000 7f8e4ace1640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e44107280 0x7f8e4419e980 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.707+0000 7f8e4ace1640 1 --2- 172.21.15.19:0/3844894375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e44102240 0x7f8e4419e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.707+0000 7f8e4ace1640 1 -- 172.21.15.19:0/3844894375 >> 172.21.15.19:0/3844894375 conn(0x7f8e440790e0 msgr2=0x7f8e441001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:51.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.707+0000 7f8e4ace1640 1 -- 172.21.15.19:0/3844894375 shutdown_connections 2024-04-25T12:17:51.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.707+0000 7f8e4ace1640 1 -- 172.21.15.19:0/3844894375 wait complete. 2024-04-25T12:17:51.719 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T12:00:20.898809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T12:05:01.658354+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T12:05:02.969078+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T12:17:51.720 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T12:17:51.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.824+0000 7f494b74f640 1 -- 172.21.15.19:0/3700142993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4944107270 msgr2=0x7f494410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:51.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.824+0000 7f494b74f640 1 --2- 172.21.15.19:0/3700142993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4944107270 0x7f494410ee40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4934009ca0 tx=0x7f493402a8c0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.824+0000 7f494b74f640 1 -- 172.21.15.19:0/3700142993 shutdown_connections 2024-04-25T12:17:51.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.824+0000 7f494b74f640 1 --2- 172.21.15.19:0/3700142993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4944107270 0x7f494410ee40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.824+0000 7f494b74f640 1 --2- 172.21.15.19:0/3700142993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f494406b1f0 0x7f4944106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.824+0000 7f494b74f640 1 --2- 172.21.15.19:0/3700142993 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f494406a840 0x7f494406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.824+0000 7f494b74f640 1 -- 172.21.15.19:0/3700142993 >> 172.21.15.19:0/3700142993 conn(0x7f4944077eb0 msgr2=0x7f49440782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:51.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.824+0000 7f494b74f640 1 -- 172.21.15.19:0/3700142993 shutdown_connections 2024-04-25T12:17:51.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.824+0000 7f494b74f640 1 -- 172.21.15.19:0/3700142993 wait complete. 2024-04-25T12:17:51.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f494b74f640 1 Processor -- start 2024-04-25T12:17:51.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f494b74f640 1 -- start start 2024-04-25T12:17:51.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f494b74f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f494406a840 0x7f49441986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:51.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f494b74f640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4944107270 0x7f4944198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:51.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f494b74f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494419cfb0 0x7f494419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:51.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f494b74f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49441122f0 con 0x7f494419cfb0 2024-04-25T12:17:51.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f494b74f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4944112170 con 0x7f494406a840 2024-04-25T12:17:51.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f494b74f640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f4944112470 con 0x7f4944107270 2024-04-25T12:17:51.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f4949cec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494419cfb0 0x7f494419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:51.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f4949cec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494419cfb0 0x7f494419d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47378/0 (socket says 172.21.15.19:47378) 2024-04-25T12:17:51.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f4949cec640 1 -- 172.21.15.19:0/223815736 learned_addr learned my addr 172.21.15.19:0/223815736 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:51.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f49494eb640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f494406a840 0x7f49441986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:51.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f4948cea640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4944107270 0x7f4944198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:51.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f4949cec640 1 -- 172.21.15.19:0/223815736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4944107270 msgr2=0x7f4944198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:51.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f4949cec640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4944107270 0x7f4944198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f4949cec640 1 -- 172.21.15.19:0/223815736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f494406a840 msgr2=0x7f49441986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:51.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f4949cec640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f494406a840 0x7f49441986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:51.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f4949cec640 1 -- 172.21.15.19:0/223815736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494419daa0 con 0x7f494419cfb0 2024-04-25T12:17:51.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.825+0000 7f4948cea640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4944107270 0x7f4944198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:17:51.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.826+0000 7f4949cec640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494419cfb0 0x7f494419d460 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4934005e30 tx=0x7f4934003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:51.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.826+0000 7f49427fc640 1 -- 172.21.15.19:0/223815736 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934031650 con 0x7f494419cfb0 2024-04-25T12:17:51.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.826+0000 7f49427fc640 1 -- 172.21.15.19:0/223815736 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f49340317f0 con 0x7f494419cfb0 2024-04-25T12:17:51.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.826+0000 7f494b74f640 1 -- 172.21.15.19:0/223815736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494419dd30 con 0x7f494419cfb0 2024-04-25T12:17:51.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.826+0000 7f49427fc640 1 -- 172.21.15.19:0/223815736 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493402fb70 con 0x7f494419cfb0 2024-04-25T12:17:51.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.826+0000 7f494b74f640 1 -- 172.21.15.19:0/223815736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49441a5550 con 0x7f494419cfb0 2024-04-25T12:17:51.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.827+0000 7f494b74f640 1 -- 172.21.15.19:0/223815736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49440691c0 con 0x7f494419cfb0 2024-04-25T12:17:51.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.828+0000 7f49427fc640 1 -- 172.21.15.19:0/223815736 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f493402fd10 con 0x7f494419cfb0 2024-04-25T12:17:51.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.829+0000 7f49427fc640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f492c0a3d20 0x7f492c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:51.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.829+0000 7f49427fc640 1 -- 172.21.15.19:0/223815736 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f49340f1e40 con 0x7f494419cfb0 2024-04-25T12:17:51.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.829+0000 7f49494eb640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f492c0a3d20 0x7f492c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:51.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.829+0000 7f49494eb640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f492c0a3d20 0x7f492c0a61e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f493c00bbe0 tx=0x7f493c00b390 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:51.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:51.832+0000 7f49427fc640 1 -- 172.21.15.19:0/223815736 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49340be560 con 0x7f494419cfb0 2024-04-25T12:17:52.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.018+0000 7f494b74f640 1 -- 172.21.15.19:0/223815736 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f494406b1f0 con 0x7f494419cfb0 2024-04-25T12:17:52.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.018+0000 7f49427fc640 1 -- 172.21.15.19:0/223815736 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f49340bde60 con 0x7f494419cfb0 2024-04-25T12:17:52.019 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-04-25T12:17:52.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.020+0000 7f494b74f640 1 -- 172.21.15.19:0/223815736 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f492c0a3d20 msgr2=0x7f492c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.020+0000 7f494b74f640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f492c0a3d20 0x7f492c0a61e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f493c00bbe0 tx=0x7f493c00b390 comp rx=0 tx=0).stop 2024-04-25T12:17:52.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.020+0000 7f494b74f640 1 -- 172.21.15.19:0/223815736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494419cfb0 msgr2=0x7f494419d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.020+0000 7f494b74f640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494419cfb0 0x7f494419d460 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4934005e30 tx=0x7f4934003f90 comp rx=0 tx=0).stop 2024-04-25T12:17:52.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.020+0000 7f494b74f640 1 -- 172.21.15.19:0/223815736 shutdown_connections 2024-04-25T12:17:52.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.020+0000 7f494b74f640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f492c0a3d20 0x7f492c0a61e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.020+0000 7f494b74f640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f494419cfb0 0x7f494419d460 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.020+0000 7f494b74f640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4944107270 0x7f4944198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.020+0000 7f494b74f640 1 --2- 172.21.15.19:0/223815736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f494406a840 0x7f49441986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.020+0000 7f494b74f640 1 -- 172.21.15.19:0/223815736 >> 172.21.15.19:0/223815736 conn(0x7f4944077eb0 msgr2=0x7f49440fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:52.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.020+0000 7f494b74f640 1 -- 172.21.15.19:0/223815736 shutdown_connections 2024-04-25T12:17:52.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.020+0000 7f494b74f640 1 -- 172.21.15.19:0/223815736 wait complete. 2024-04-25T12:17:52.032 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T12:17:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.136+0000 7f9876dad640 1 -- 172.21.15.19:0/78023973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987006a5b0 msgr2=0x7f987006aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.136+0000 7f9876dad640 1 --2- 172.21.15.19:0/78023973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987006a5b0 0x7f987006aa30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9864009930 tx=0x7f986402a420 comp rx=0 tx=0).stop 2024-04-25T12:17:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.136+0000 7f9876dad640 1 -- 172.21.15.19:0/78023973 shutdown_connections 2024-04-25T12:17:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.136+0000 7f9876dad640 1 --2- 172.21.15.19:0/78023973 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f987006af70 0x7f987010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.136+0000 7f9876dad640 1 --2- 172.21.15.19:0/78023973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987006a5b0 0x7f987006aa30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.136+0000 7f9876dad640 1 --2- 172.21.15.19:0/78023973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9870108060 0x7f9870108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.136+0000 7f9876dad640 1 -- 172.21.15.19:0/78023973 >> 172.21.15.19:0/78023973 conn(0x7f9870077c40 msgr2=0x7f9870078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:52.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.136+0000 7f9876dad640 1 -- 172.21.15.19:0/78023973 shutdown_connections 2024-04-25T12:17:52.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.136+0000 7f9876dad640 1 -- 172.21.15.19:0/78023973 wait complete. 2024-04-25T12:17:52.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.137+0000 7f9876dad640 1 Processor -- start 2024-04-25T12:17:52.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.137+0000 7f9876dad640 1 -- start start 2024-04-25T12:17:52.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.137+0000 7f9876dad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987006a5b0 0x7f9870198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:52.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.137+0000 7f9876dad640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f987006af70 0x7f9870198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:52.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.137+0000 7f9876dad640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9870108060 0x7f987019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:52.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.137+0000 7f9876dad640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9870112300 con 0x7f987006a5b0 2024-04-25T12:17:52.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.137+0000 7f9876dad640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9870112180 con 0x7f9870108060 2024-04-25T12:17:52.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.137+0000 7f9876dad640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f9870112480 con 0x7f987006af70 2024-04-25T12:17:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.137+0000 7f9874b49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987006a5b0 0x7f9870198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.137+0000 7f9874b49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987006a5b0 0x7f9870198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47386/0 (socket says 172.21.15.19:47386) 2024-04-25T12:17:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.137+0000 7f9874b49640 1 -- 172.21.15.19:0/3225512391 learned_addr learned my addr 172.21.15.19:0/3225512391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.137+0000 7f986ffff640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f987006af70 0x7f9870198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.138+0000 7f987534a640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9870108060 0x7f987019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.138+0000 7f9874b49640 1 -- 172.21.15.19:0/3225512391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f987006af70 msgr2=0x7f9870198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.138+0000 7f9874b49640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f987006af70 0x7f9870198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.138+0000 7f9874b49640 1 -- 172.21.15.19:0/3225512391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9870108060 msgr2=0x7f987019d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.138+0000 7f9874b49640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9870108060 0x7f987019d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.138+0000 7f9874b49640 1 -- 172.21.15.19:0/3225512391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987019d730 con 0x7f987006a5b0 2024-04-25T12:17:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.138+0000 7f987534a640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9870108060 0x7f987019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:17:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.138+0000 7f9874b49640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987006a5b0 0x7f9870198740 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f986000c890 tx=0x7f986000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.138+0000 7f986dffb640 1 -- 172.21.15.19:0/3225512391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860004220 con 0x7f987006a5b0 2024-04-25T12:17:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.138+0000 7f9876dad640 1 -- 172.21.15.19:0/3225512391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987019da20 con 0x7f987006a5b0 2024-04-25T12:17:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.138+0000 7f986dffb640 1 -- 172.21.15.19:0/3225512391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f98600043c0 con 0x7f987006a5b0 2024-04-25T12:17:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.138+0000 7f986dffb640 1 -- 172.21.15.19:0/3225512391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98600049e0 con 0x7f987006a5b0 2024-04-25T12:17:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.138+0000 7f9876dad640 1 -- 172.21.15.19:0/3225512391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98701a52b0 con 0x7f987006a5b0 2024-04-25T12:17:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.139+0000 7f9876dad640 1 -- 172.21.15.19:0/3225512391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9870103650 con 0x7f987006a5b0 2024-04-25T12:17:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.140+0000 7f986dffb640 1 -- 172.21.15.19:0/3225512391 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9860005140 con 0x7f987006a5b0 2024-04-25T12:17:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.141+0000 7f986dffb640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f98580a3d20 0x7f98580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.141+0000 7f986ffff640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f98580a3d20 0x7f98580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.141+0000 7f986dffb640 1 -- 172.21.15.19:0/3225512391 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f98600d0920 con 0x7f987006a5b0 2024-04-25T12:17:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.142+0000 7f986ffff640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f98580a3d20 0x7f98580a61e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9864009900 tx=0x7f98640058d0 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.144+0000 7f986dffb640 1 -- 172.21.15.19:0/3225512391 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9860014030 con 0x7f987006a5b0 2024-04-25T12:17:52.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.331+0000 7f9876dad640 1 -- 172.21.15.19:0/3225512391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f98701a5510 con 0x7f987006a5b0 2024-04-25T12:17:52.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.332+0000 7f986dffb640 1 -- 172.21.15.19:0/3225512391 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f986009cf90 con 0x7f987006a5b0 2024-04-25T12:17:52.333 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-04-25T12:17:52.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.334+0000 7f9876dad640 1 -- 172.21.15.19:0/3225512391 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f98580a3d20 msgr2=0x7f98580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.334+0000 7f9876dad640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f98580a3d20 0x7f98580a61e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9864009900 tx=0x7f98640058d0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.334+0000 7f9876dad640 1 -- 172.21.15.19:0/3225512391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987006a5b0 msgr2=0x7f9870198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.334+0000 7f9876dad640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987006a5b0 0x7f9870198740 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f986000c890 tx=0x7f986000cd50 comp rx=0 tx=0).stop 2024-04-25T12:17:52.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.334+0000 7f9876dad640 1 -- 172.21.15.19:0/3225512391 shutdown_connections 2024-04-25T12:17:52.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.334+0000 7f9876dad640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f98580a3d20 0x7f98580a61e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.334+0000 7f9876dad640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9870108060 0x7f987019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.334+0000 7f9876dad640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f987006af70 0x7f9870198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.334+0000 7f9876dad640 1 --2- 172.21.15.19:0/3225512391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f987006a5b0 0x7f9870198740 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.334+0000 7f9876dad640 1 -- 172.21.15.19:0/3225512391 >> 172.21.15.19:0/3225512391 conn(0x7f9870077c40 msgr2=0x7f98700ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.334+0000 7f9876dad640 1 -- 172.21.15.19:0/3225512391 shutdown_connections 2024-04-25T12:17:52.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.334+0000 7f9876dad640 1 -- 172.21.15.19:0/3225512391 wait complete. 2024-04-25T12:17:52.345 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T12:17:52.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.449+0000 7f9452a59640 1 -- 172.21.15.19:0/533359040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c074220 msgr2=0x7f944c074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.449+0000 7f9452a59640 1 --2- 172.21.15.19:0/533359040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c074220 0x7f944c074600 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f943c009900 tx=0x7f943c02a480 comp rx=0 tx=0).stop 2024-04-25T12:17:52.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.449+0000 7f9452a59640 1 -- 172.21.15.19:0/533359040 shutdown_connections 2024-04-25T12:17:52.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.449+0000 7f9452a59640 1 --2- 172.21.15.19:0/533359040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f944c10e830 0x7f944c110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.449+0000 7f9452a59640 1 --2- 172.21.15.19:0/533359040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f944c074b40 0x7f944c10e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.449+0000 7f9452a59640 1 --2- 172.21.15.19:0/533359040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c074220 0x7f944c074600 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.449+0000 7f9452a59640 1 -- 172.21.15.19:0/533359040 >> 172.21.15.19:0/533359040 conn(0x7f944c0793a0 msgr2=0x7f944c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:52.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.450+0000 7f9452a59640 1 -- 172.21.15.19:0/533359040 shutdown_connections 2024-04-25T12:17:52.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.450+0000 7f9452a59640 1 -- 172.21.15.19:0/533359040 wait complete. 2024-04-25T12:17:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.450+0000 7f9452a59640 1 Processor -- start 2024-04-25T12:17:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.450+0000 7f9452a59640 1 -- start start 2024-04-25T12:17:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.450+0000 7f9452a59640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f944c074220 0x7f944c1a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.450+0000 7f9452a59640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f944c074b40 0x7f944c1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9452a59640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c10e830 0x7f944c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9452a59640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f944c114520 con 0x7f944c10e830 2024-04-25T12:17:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9452a59640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1143a0 con 0x7f944c074220 2024-04-25T12:17:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9452a59640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1146a0 con 0x7f944c074b40 2024-04-25T12:17:52.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9450ff6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c10e830 0x7f944c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:52.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9450ff6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c10e830 0x7f944c19a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47404/0 (socket says 172.21.15.19:47404) 2024-04-25T12:17:52.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9450ff6640 1 -- 172.21.15.19:0/3241043081 learned_addr learned my addr 172.21.15.19:0/3241043081 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:52.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f944b7fe640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f944c074b40 0x7f944c1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:52.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f944bfff640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f944c074220 0x7f944c1a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:52.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9450ff6640 1 -- 172.21.15.19:0/3241043081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f944c074b40 msgr2=0x7f944c1a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9450ff6640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f944c074b40 0x7f944c1a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9450ff6640 1 -- 172.21.15.19:0/3241043081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f944c074220 msgr2=0x7f944c1a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9450ff6640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f944c074220 0x7f944c1a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9450ff6640 1 -- 172.21.15.19:0/3241043081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c19b0a0 con 0x7f944c10e830 2024-04-25T12:17:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f944b7fe640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f944c074b40 0x7f944c1a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:17:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9450ff6640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c10e830 0x7f944c19a920 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f944400cab0 tx=0x7f944400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f94497fa640 1 -- 172.21.15.19:0/3241043081 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944400ae20 con 0x7f944c10e830 2024-04-25T12:17:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9452a59640 1 -- 172.21.15.19:0/3241043081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944c19b390 con 0x7f944c10e830 2024-04-25T12:17:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f94497fa640 1 -- 172.21.15.19:0/3241043081 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f944400f920 con 0x7f944c10e830 2024-04-25T12:17:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f94497fa640 1 -- 172.21.15.19:0/3241043081 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444018b40 con 0x7f944c10e830 2024-04-25T12:17:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.451+0000 7f9452a59640 1 -- 172.21.15.19:0/3241043081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f944c1a75e0 con 0x7f944c10e830 2024-04-25T12:17:52.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.453+0000 7f9452a59640 1 -- 172.21.15.19:0/3241043081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f944c0691c0 con 0x7f944c10e830 2024-04-25T12:17:52.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.454+0000 7f94497fa640 1 -- 172.21.15.19:0/3241043081 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f944400fad0 con 0x7f944c10e830 2024-04-25T12:17:52.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.454+0000 7f94497fa640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f94340a3da0 0x7f94340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:52.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.454+0000 7f94497fa640 1 -- 172.21.15.19:0/3241043081 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f94440d1030 con 0x7f944c10e830 2024-04-25T12:17:52.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.454+0000 7f944bfff640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f94340a3da0 0x7f94340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:52.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.455+0000 7f944bfff640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f94340a3da0 0x7f94340a6260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f943c000c00 tx=0x7f943c005a20 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:52.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.458+0000 7f94497fa640 1 -- 172.21.15.19:0/3241043081 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9444014030 con 0x7f944c10e830 2024-04-25T12:17:52.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.644+0000 7f9452a59640 1 -- 172.21.15.19:0/3241043081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f944c02d5c0 con 0x7f944c10e830 2024-04-25T12:17:52.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.645+0000 7f94497fa640 1 -- 172.21.15.19:0/3241043081 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f944409d720 con 0x7f944c10e830 2024-04-25T12:17:52.646 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-04-25T12:17:52.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.647+0000 7f9452a59640 1 -- 172.21.15.19:0/3241043081 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f94340a3da0 msgr2=0x7f94340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.647+0000 7f9452a59640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f94340a3da0 0x7f94340a6260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f943c000c00 tx=0x7f943c005a20 comp rx=0 tx=0).stop 2024-04-25T12:17:52.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.647+0000 7f9452a59640 1 -- 172.21.15.19:0/3241043081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c10e830 msgr2=0x7f944c19a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.647+0000 7f9452a59640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c10e830 0x7f944c19a920 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f944400cab0 tx=0x7f944400cf70 comp rx=0 tx=0).stop 2024-04-25T12:17:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.647+0000 7f9452a59640 1 -- 172.21.15.19:0/3241043081 shutdown_connections 2024-04-25T12:17:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.647+0000 7f9452a59640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f94340a3da0 0x7f94340a6260 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.647+0000 7f9452a59640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f944c10e830 0x7f944c19a920 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.647+0000 7f9452a59640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f944c074b40 0x7f944c1a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.647+0000 7f9452a59640 1 --2- 172.21.15.19:0/3241043081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f944c074220 0x7f944c1a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.647+0000 7f9452a59640 1 -- 172.21.15.19:0/3241043081 >> 172.21.15.19:0/3241043081 conn(0x7f944c0793a0 msgr2=0x7f944c0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.647+0000 7f9452a59640 1 -- 172.21.15.19:0/3241043081 shutdown_connections 2024-04-25T12:17:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.647+0000 7f9452a59640 1 -- 172.21.15.19:0/3241043081 wait complete. 2024-04-25T12:17:52.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:52 smithi019 ceph-mon[29213]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:52.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:52 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3844894375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:17:52.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:52 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/223815736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:17:52.659 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T12:17:52.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:52 smithi169 ceph-mon[32710]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:52.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:52 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3844894375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:17:52.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:52 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/223815736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:17:52.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:52 smithi150 ceph-mon[32250]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T12:17:52.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:52 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3844894375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:17:52.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:52 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/223815736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T12:17:52.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.765+0000 7f62fe119640 1 -- 172.21.15.19:0/4099900967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f8105e70 msgr2=0x7f62f8106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.765+0000 7f62fe119640 1 --2- 172.21.15.19:0/4099900967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f8105e70 0x7f62f8106250 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009900 tx=0x7f62e802a470 comp rx=0 tx=0).stop 2024-04-25T12:17:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.765+0000 7f62fe119640 1 -- 172.21.15.19:0/4099900967 shutdown_connections 2024-04-25T12:17:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.765+0000 7f62fe119640 1 --2- 172.21.15.19:0/4099900967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62f8101aa0 0x7f62f810dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.765+0000 7f62fe119640 1 --2- 172.21.15.19:0/4099900967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62f81010e0 0x7f62f8101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.765+0000 7f62fe119640 1 --2- 172.21.15.19:0/4099900967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f8105e70 0x7f62f8106250 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.765+0000 7f62fe119640 1 -- 172.21.15.19:0/4099900967 >> 172.21.15.19:0/4099900967 conn(0x7f62f8077cb0 msgr2=0x7f62f80780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.765+0000 7f62fe119640 1 -- 172.21.15.19:0/4099900967 shutdown_connections 2024-04-25T12:17:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.765+0000 7f62fe119640 1 -- 172.21.15.19:0/4099900967 wait complete. 2024-04-25T12:17:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.765+0000 7f62fe119640 1 Processor -- start 2024-04-25T12:17:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62fe119640 1 -- start start 2024-04-25T12:17:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62fe119640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f81010e0 0x7f62f8199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62fe119640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62f8101aa0 0x7f62f8199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62fe119640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62f8105e70 0x7f62f81a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62fe119640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8111290 con 0x7f62f81010e0 2024-04-25T12:17:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62fe119640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8111110 con 0x7f62f8101aa0 2024-04-25T12:17:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62fe119640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8111410 con 0x7f62f8105e70 2024-04-25T12:17:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62f77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f81010e0 0x7f62f8199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62f77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f81010e0 0x7f62f8199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47424/0 (socket says 172.21.15.19:47424) 2024-04-25T12:17:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62f77fe640 1 -- 172.21.15.19:0/3414355690 learned_addr learned my addr 172.21.15.19:0/3414355690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62f6ffd640 1 --2- 172.21.15.19:0/3414355690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62f8101aa0 0x7f62f8199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62f7fff640 1 --2- 172.21.15.19:0/3414355690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62f8105e70 0x7f62f81a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62f77fe640 1 -- 172.21.15.19:0/3414355690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62f8105e70 msgr2=0x7f62f81a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62f77fe640 1 --2- 172.21.15.19:0/3414355690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62f8105e70 0x7f62f81a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62f77fe640 1 -- 172.21.15.19:0/3414355690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62f8101aa0 msgr2=0x7f62f8199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62f77fe640 1 --2- 172.21.15.19:0/3414355690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62f8101aa0 0x7f62f8199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62f77fe640 1 -- 172.21.15.19:0/3414355690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f81a4c50 con 0x7f62f81010e0 2024-04-25T12:17:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.766+0000 7f62f77fe640 1 --2- 172.21.15.19:0/3414355690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f81010e0 0x7f62f8199690 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f62e8005940 tx=0x7f62e8004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.767+0000 7f62f4ff9640 1 -- 172.21.15.19:0/3414355690 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e803f030 con 0x7f62f81010e0 2024-04-25T12:17:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.767+0000 7f62f4ff9640 1 -- 172.21.15.19:0/3414355690 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f62e80054d0 con 0x7f62f81010e0 2024-04-25T12:17:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.767+0000 7f62fe119640 1 -- 172.21.15.19:0/3414355690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f81a4e20 con 0x7f62f81010e0 2024-04-25T12:17:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.767+0000 7f62f4ff9640 1 -- 172.21.15.19:0/3414355690 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8035070 con 0x7f62f81010e0 2024-04-25T12:17:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.767+0000 7f62fe119640 1 -- 172.21.15.19:0/3414355690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f81a5280 con 0x7f62f81010e0 2024-04-25T12:17:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.768+0000 7f62fe119640 1 -- 172.21.15.19:0/3414355690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f8068d70 con 0x7f62f81010e0 2024-04-25T12:17:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.769+0000 7f62f4ff9640 1 -- 172.21.15.19:0/3414355690 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f62e8042030 con 0x7f62f81010e0 2024-04-25T12:17:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.769+0000 7f62f4ff9640 1 --2- 172.21.15.19:0/3414355690 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f62e00a3da0 0x7f62e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.770+0000 7f62f4ff9640 1 -- 172.21.15.19:0/3414355690 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f62e80f1160 con 0x7f62f81010e0 2024-04-25T12:17:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.770+0000 7f62f6ffd640 1 --2- 172.21.15.19:0/3414355690 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f62e00a3da0 0x7f62e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.770+0000 7f62f6ffd640 1 --2- 172.21.15.19:0/3414355690 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f62e00a3da0 0x7f62e00a6260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f62ec005fd0 tx=0x7f62ec005850 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.773+0000 7f62f4ff9640 1 -- 172.21.15.19:0/3414355690 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62e80bd7d0 con 0x7f62f81010e0 2024-04-25T12:17:52.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.974+0000 7f62fe119640 1 -- 172.21.15.19:0/3414355690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f62f80696d0 con 0x7f62f81010e0 2024-04-25T12:17:52.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.974+0000 7f62f4ff9640 1 -- 172.21.15.19:0/3414355690 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v96) ==== 81+0+940 (secure 0 0 0) 0x7f62e8004770 con 0x7f62f81010e0 2024-04-25T12:17:52.975 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2813829603 2024-04-26T12:07:09.072296+0000 2024-04-25T12:17:52.976 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1048024792 2024-04-26T12:07:09.072296+0000 2024-04-25T12:17:52.976 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2853466745 2024-04-26T12:07:09.072296+0000 2024-04-25T12:17:52.976 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1094943085 2024-04-26T12:07:09.072296+0000 2024-04-25T12:17:52.976 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/37557166 2024-04-26T11:57:54.811890+0000 2024-04-25T12:17:52.976 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2734487323 2024-04-26T12:07:09.072296+0000 2024-04-25T12:17:52.976 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/963006875 2024-04-26T11:57:54.811890+0000 2024-04-25T12:17:52.976 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/963006875 2024-04-26T11:57:54.811890+0000 2024-04-25T12:17:52.977 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2853466745 2024-04-26T12:07:09.072296+0000 2024-04-25T12:17:52.977 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/556620573 2024-04-26T11:57:27.561732+0000 2024-04-25T12:17:52.977 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/556620573 2024-04-26T11:57:27.561732+0000 2024-04-25T12:17:52.977 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1079165232 2024-04-26T11:57:54.811890+0000 2024-04-25T12:17:52.977 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/4104025927 2024-04-26T11:57:27.561732+0000 2024-04-25T12:17:52.977 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2968863069 2024-04-26T11:57:54.811890+0000 2024-04-25T12:17:52.977 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3693313603 2024-04-26T11:57:27.561732+0000 2024-04-25T12:17:52.977 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1919062753 2024-04-26T11:57:27.561732+0000 2024-04-25T12:17:52.978 INFO:teuthology.orchestra.run.smithi019.stderr:listed 16 entries 2024-04-25T12:17:52.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.976+0000 7f62fe119640 1 -- 172.21.15.19:0/3414355690 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f62e00a3da0 msgr2=0x7f62e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.976+0000 7f62fe119640 1 --2- 172.21.15.19:0/3414355690 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f62e00a3da0 0x7f62e00a6260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f62ec005fd0 tx=0x7f62ec005850 comp rx=0 tx=0).stop 2024-04-25T12:17:52.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.976+0000 7f62fe119640 1 -- 172.21.15.19:0/3414355690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f81010e0 msgr2=0x7f62f8199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:52.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.976+0000 7f62fe119640 1 --2- 172.21.15.19:0/3414355690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f81010e0 0x7f62f8199690 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f62e8005940 tx=0x7f62e8004c00 comp rx=0 tx=0).stop 2024-04-25T12:17:52.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.976+0000 7f62fe119640 1 -- 172.21.15.19:0/3414355690 shutdown_connections 2024-04-25T12:17:52.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.976+0000 7f62fe119640 1 --2- 172.21.15.19:0/3414355690 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f62e00a3da0 0x7f62e00a6260 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.976+0000 7f62fe119640 1 --2- 172.21.15.19:0/3414355690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62f8105e70 0x7f62f81a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.976+0000 7f62fe119640 1 --2- 172.21.15.19:0/3414355690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62f8101aa0 0x7f62f8199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.976+0000 7f62fe119640 1 --2- 172.21.15.19:0/3414355690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62f81010e0 0x7f62f8199690 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:52.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.976+0000 7f62fe119640 1 -- 172.21.15.19:0/3414355690 >> 172.21.15.19:0/3414355690 conn(0x7f62f8077cb0 msgr2=0x7f62f80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:52.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.977+0000 7f62fe119640 1 -- 172.21.15.19:0/3414355690 shutdown_connections 2024-04-25T12:17:52.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:52.977+0000 7f62fe119640 1 -- 172.21.15.19:0/3414355690 wait complete. 2024-04-25T12:17:52.988 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T12:17:52.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:17:52.988 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T12:17:53.021 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T12:17:53.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:17:53.022 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T12:17:53.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:17:53.087 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T12:17:53.142 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T12:17:53.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:17:53.142 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T12:17:53.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:17:53.208 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T12:17:53.301 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T12:17:53.302 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-04-25T12:17:53.302 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T12:17:53.318 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=282 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/c48c1744792c9955d90208b677d4862d3b24010d883dea5b04962595f0dbf160/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/NU4K7HLVX6NLTS7EWBVQVF54BK:/var/lib/containers/storage/overlay/l/JARFXB6DTRJJAFYDMGRISQXFEY,upperdir=/var/lib/containers/storage/overlay/c48c1744792c9955d90208b677d4862d3b24010d883dea5b04962595f0dbf160/diff,workdir=/var/lib/containers/storage/overlay/c48c1744792c9955d90208b677d4862d3b24010d883dea5b04962595f0dbf160/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/88ca4b2482d719c5795b26fa9e88d64a086477adeea2ecb150604b86e6beaf86/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/NU4K7HLVX6NLTS7EWBVQVF54BK:/var/lib/containers/storage/overlay/l/JARFXB6DTRJJAFYDMGRISQXFEY,upperdir=/var/lib/containers/storage/overlay/88ca4b2482d719c5795b26fa9e88d64a086477adeea2ecb150604b86e6beaf86/diff,workdir=/var/lib/containers/storage/overlay/88ca4b2482d719c5795b26fa9e88d64a086477adeea2ecb150604b86e6beaf86/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/235163f80be955f013a39af7509d9399bec7f1bba3b4d9b44f1c7fd0f48d4dc0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/NU4K7HLVX6NLTS7EWBVQVF54BK:/var/lib/containers/storage/overlay/l/JARFXB6DTRJJAFYDMGRISQXFEY,upperdir=/var/lib/containers/storage/overlay/235163f80be955f013a39af7509d9399bec7f1bba3b4d9b44f1c7fd0f48d4dc0/diff,workdir=/var/lib/containers/storage/overlay/235163f80be955f013a39af7509d9399bec7f1bba3b4d9b44f1c7fd0f48d4dc0/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8caf0bc1be72bb398203d78054f33d7f655b96c475e2fbba95de99809af4c96a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/NU4K7HLVX6NLTS7EWBVQVF54BK:/var/lib/containers/storage/overlay/l/JARFXB6DTRJJAFYDMGRISQXFEY,upperdir=/var/lib/containers/storage/overlay/8caf0bc1be72bb398203d78054f33d7f655b96c475e2fbba95de99809af4c96a/diff,workdir=/var/lib/containers/storage/overlay/8caf0bc1be72bb398203d78054f33d7f655b96c475e2fbba95de99809af4c96a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/fd6c0ee1c868effaf78aeead75b7fe59b4a04f29723218050e42475a80de8541/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/NU4K7HLVX6NLTS7EWBVQVF54BK:/var/lib/containers/storage/overlay/l/JARFXB6DTRJJAFYDMGRISQXFEY,upperdir=/var/lib/containers/storage/overlay/fd6c0ee1c868effaf78aeead75b7fe59b4a04f29723218050e42475a80de8541/diff,workdir=/var/lib/containers/storage/overlay/fd6c0ee1c868effaf78aeead75b7fe59b4a04f29723218050e42475a80de8541/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f403abf8f67751ea1aaea586a7256b084b98e46dbc2cd0c78198168671bac9f6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/NU4K7HLVX6NLTS7EWBVQVF54BK:/var/lib/containers/storage/overlay/l/JARFXB6DTRJJAFYDMGRISQXFEY,upperdir=/var/lib/containers/storage/overlay/f403abf8f67751ea1aaea586a7256b084b98e46dbc2cd0c78198168671bac9f6/diff,workdir=/var/lib/containers/storage/overlay/f403abf8f67751ea1aaea586a7256b084b98e46dbc2cd0c78198168671bac9f6/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9087ba03ab2cbd7bb3ffaa7aec9037dc1c871211f7d78a90749e556c5310e3fc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/NU4K7HLVX6NLTS7EWBVQVF54BK:/var/lib/containers/storage/overlay/l/JARFXB6DTRJJAFYDMGRISQXFEY,upperdir=/var/lib/containers/storage/overlay/9087ba03ab2cbd7bb3ffaa7aec9037dc1c871211f7d78a90749e556c5310e3fc/diff,workdir=/var/lib/containers/storage/overlay/9087ba03ab2cbd7bb3ffaa7aec9037dc1c871211f7d78a90749e556c5310e3fc/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4388fba53e1512738ce020c1f05e9a7a1d106a113309d7513cbd8d9b1919afac/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/NU4K7HLVX6NLTS7EWBVQVF54BK:/var/lib/containers/storage/overlay/l/JARFXB6DTRJJAFYDMGRISQXFEY,upperdir=/var/lib/containers/storage/overlay/4388fba53e1512738ce020c1f05e9a7a1d106a113309d7513cbd8d9b1919afac/diff,workdir=/var/lib/containers/storage/overlay/4388fba53e1512738ce020c1f05e9a7a1d106a113309d7513cbd8d9b1919afac/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/433bb8886ebb308c252818c27d8da2502c3d8bdd79b72634ced1cd4f43de4c1d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/NU4K7HLVX6NLTS7EWBVQVF54BK:/var/lib/containers/storage/overlay/l/JARFXB6DTRJJAFYDMGRISQXFEY,upperdir=/var/lib/containers/storage/overlay/433bb8886ebb308c252818c27d8da2502c3d8bdd79b72634ced1cd4f43de4c1d/diff,workdir=/var/lib/containers/storage/overlay/433bb8886ebb308c252818c27d8da2502c3d8bdd79b72634ced1cd4f43de4c1d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7881c98f6dccf40d6aa43ae2fec3f30d73c7b42b13d2c74157850b049fe2cdd3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/NU4K7HLVX6NLTS7EWBVQVF54BK:/var/lib/containers/storage/overlay/l/JARFXB6DTRJJAFYDMGRISQXFEY,upperdir=/var/lib/containers/storage/overlay/7881c98f6dccf40d6aa43ae2fec3f30d73c7b42b13d2c74157850b049fe2cdd3/diff,workdir=/var/lib/containers/storage/overlay/7881c98f6dccf40d6aa43ae2fec3f30d73c7b42b13d2c74157850b049fe2cdd3/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-25T12:17:53.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:17:53.319 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-04-25T12:17:53.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T12:17:53.374 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-04-25T12:17:53.441 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-04-25T12:17:53.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:17:53.443 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T12:17:53.454 INFO:tasks.cephadm:Teardown begin 2024-04-25T12:17:53.455 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T12:17:53.455 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:17:53.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:53 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3225512391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:17:53.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:53 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3241043081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:17:53.551 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:53 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/3414355690' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T12:17:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.587+0000 7fb435929640 1 -- 172.21.15.19:0/1270619839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb430102bf0 msgr2=0x7fb430106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.587+0000 7fb435929640 1 --2- 172.21.15.19:0/1270619839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb430102bf0 0x7fb430106ba0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb418009900 tx=0x7fb41802a410 comp rx=0 tx=0).stop 2024-04-25T12:17:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.587+0000 7fb435929640 1 -- 172.21.15.19:0/1270619839 shutdown_connections 2024-04-25T12:17:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.587+0000 7fb435929640 1 --2- 172.21.15.19:0/1270619839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb430107280 0x7fb43010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.587+0000 7fb435929640 1 --2- 172.21.15.19:0/1270619839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb430102bf0 0x7fb430106ba0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.587+0000 7fb435929640 1 --2- 172.21.15.19:0/1270619839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb430102240 0x7fb430102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.587+0000 7fb435929640 1 -- 172.21.15.19:0/1270619839 >> 172.21.15.19:0/1270619839 conn(0x7fb4300790e0 msgr2=0x7fb4300794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.587+0000 7fb435929640 1 -- 172.21.15.19:0/1270619839 shutdown_connections 2024-04-25T12:17:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.587+0000 7fb435929640 1 -- 172.21.15.19:0/1270619839 wait complete. 2024-04-25T12:17:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.587+0000 7fb435929640 1 Processor -- start 2024-04-25T12:17:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.587+0000 7fb435929640 1 -- start start 2024-04-25T12:17:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb435929640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb430102240 0x7fb43019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb435929640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb430107280 0x7fb43019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb435929640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb430198640 0x7fb430198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb435929640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb430112300 con 0x7fb430102240 2024-04-25T12:17:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb435929640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb430112180 con 0x7fb430198640 2024-04-25T12:17:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb435929640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fb430112480 con 0x7fb430107280 2024-04-25T12:17:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb42effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb430102240 0x7fb43019e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb42effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb430102240 0x7fb43019e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47432/0 (socket says 172.21.15.19:47432) 2024-04-25T12:17:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb42effd640 1 -- 172.21.15.19:0/2269825021 learned_addr learned my addr 172.21.15.19:0/2269825021 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb427fff640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb430107280 0x7fb43019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb42f7fe640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb430198640 0x7fb430198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb42effd640 1 -- 172.21.15.19:0/2269825021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb430107280 msgr2=0x7fb43019e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb42effd640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb430107280 0x7fb43019e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb42effd640 1 -- 172.21.15.19:0/2269825021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb430198640 msgr2=0x7fb430198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb42effd640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb430198640 0x7fb430198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb42effd640 1 -- 172.21.15.19:0/2269825021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb430199160 con 0x7fb430102240 2024-04-25T12:17:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb427fff640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb430107280 0x7fb43019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:17:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb42f7fe640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb430198640 0x7fb430198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:17:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb42effd640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb430102240 0x7fb43019e440 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb42000c920 tx=0x7fb42000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb434927640 1 -- 172.21.15.19:0/2269825021 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420007c70 con 0x7fb430102240 2024-04-25T12:17:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb434927640 1 -- 172.21.15.19:0/2269825021 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fb420007e10 con 0x7fb430102240 2024-04-25T12:17:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb434927640 1 -- 172.21.15.19:0/2269825021 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4200188c0 con 0x7fb430102240 2024-04-25T12:17:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb435929640 1 -- 172.21.15.19:0/2269825021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb430199450 con 0x7fb430102240 2024-04-25T12:17:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.588+0000 7fb435929640 1 -- 172.21.15.19:0/2269825021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4301a56a0 con 0x7fb430102240 2024-04-25T12:17:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.589+0000 7fb435929640 1 -- 172.21.15.19:0/2269825021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4300691c0 con 0x7fb430102240 2024-04-25T12:17:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.590+0000 7fb434927640 1 -- 172.21.15.19:0/2269825021 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb420018a60 con 0x7fb430102240 2024-04-25T12:17:53.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.591+0000 7fb434927640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb41c0a3da0 0x7fb41c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.591+0000 7fb434927640 1 -- 172.21.15.19:0/2269825021 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fb4200d09b0 con 0x7fb430102240 2024-04-25T12:17:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.591+0000 7fb427fff640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb41c0a3da0 0x7fb41c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.591+0000 7fb427fff640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb41c0a3da0 0x7fb41c0a6260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb430101b60 tx=0x7fb418032c10 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.595+0000 7fb434927640 1 -- 172.21.15.19:0/2269825021 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb420014030 con 0x7fb430102240 2024-04-25T12:17:53.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:53 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3225512391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:17:53.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:53 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3241043081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:17:53.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:53 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/3414355690' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T12:17:53.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:53 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3225512391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T12:17:53.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:53 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3241043081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T12:17:53.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:53 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/3414355690' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T12:17:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.823+0000 7fb435929640 1 -- 172.21.15.19:0/2269825021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb430199b20 con 0x7fb430102240 2024-04-25T12:17:53.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.824+0000 7fb434927640 1 -- 172.21.15.19:0/2269825021 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7fb42000de20 con 0x7fb430102240 2024-04-25T12:17:53.825 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:17:53.826 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T12:05:04.008679+0000","modified":"2024-04-25T12:07:07.722007+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24514,"mds_1":24500,"mds_2":24496,"mds_3":24484,"mds_4":14655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/1694897630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1694897630},{"type":"v1","addr":"172.21.15.19:6839","nonce":1694897630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6841/3247157172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3247157172},{"type":"v1","addr":"172.21.15.19:6841","nonce":3247157172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/3918801450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3918801450},{"type":"v1","addr":"172.21.15.150:6837","nonce":3918801450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2553528303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2553528303},{"type":"v1","addr":"172.21.15.150:6839","nonce":2553528303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/1723439500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1723439500},{"type":"v1","addr":"172.21.15.169:6839","nonce":1723439500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[14655,24484,24496,24500,24514]},"id":1}]} 2024-04-25T12:17:53.826 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-25T12:17:53.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.826+0000 7fb435929640 1 -- 172.21.15.19:0/2269825021 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb41c0a3da0 msgr2=0x7fb41c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:53.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.826+0000 7fb435929640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb41c0a3da0 0x7fb41c0a6260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb430101b60 tx=0x7fb418032c10 comp rx=0 tx=0).stop 2024-04-25T12:17:53.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.826+0000 7fb435929640 1 -- 172.21.15.19:0/2269825021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb430102240 msgr2=0x7fb43019e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:53.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.826+0000 7fb435929640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb430102240 0x7fb43019e440 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb42000c920 tx=0x7fb42000cde0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.826+0000 7fb435929640 1 -- 172.21.15.19:0/2269825021 shutdown_connections 2024-04-25T12:17:53.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.826+0000 7fb435929640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fb41c0a3da0 0x7fb41c0a6260 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.826+0000 7fb435929640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb430198640 0x7fb430198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.826+0000 7fb435929640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb430107280 0x7fb43019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.826+0000 7fb435929640 1 --2- 172.21.15.19:0/2269825021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb430102240 0x7fb43019e440 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.826+0000 7fb435929640 1 -- 172.21.15.19:0/2269825021 >> 172.21.15.19:0/2269825021 conn(0x7fb4300790e0 msgr2=0x7fb4301001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:53.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.826+0000 7fb435929640 1 -- 172.21.15.19:0/2269825021 shutdown_connections 2024-04-25T12:17:53.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.826+0000 7fb435929640 1 -- 172.21.15.19:0/2269825021 wait complete. 2024-04-25T12:17:53.838 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T12:17:53.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.939+0000 7fde7cb3d640 1 -- 172.21.15.19:0/241049173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde78102bf0 msgr2=0x7fde78106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:53.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.939+0000 7fde7cb3d640 1 --2- 172.21.15.19:0/241049173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde78102bf0 0x7fde78106ba0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fde60009900 tx=0x7fde6002a410 comp rx=0 tx=0).stop 2024-04-25T12:17:53.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.939+0000 7fde7cb3d640 1 -- 172.21.15.19:0/241049173 shutdown_connections 2024-04-25T12:17:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.939+0000 7fde7cb3d640 1 --2- 172.21.15.19:0/241049173 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde78107280 0x7fde7810ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.939+0000 7fde7cb3d640 1 --2- 172.21.15.19:0/241049173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde78102bf0 0x7fde78106ba0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.939+0000 7fde7cb3d640 1 --2- 172.21.15.19:0/241049173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde78102240 0x7fde78102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.939+0000 7fde7cb3d640 1 -- 172.21.15.19:0/241049173 >> 172.21.15.19:0/241049173 conn(0x7fde780790e0 msgr2=0x7fde780794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.939+0000 7fde7cb3d640 1 -- 172.21.15.19:0/241049173 shutdown_connections 2024-04-25T12:17:53.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.939+0000 7fde7cb3d640 1 -- 172.21.15.19:0/241049173 wait complete. 2024-04-25T12:17:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7cb3d640 1 Processor -- start 2024-04-25T12:17:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7cb3d640 1 -- start start 2024-04-25T12:17:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7cb3d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde78102240 0x7fde7819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7cb3d640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde78107280 0x7fde7819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7cb3d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde78198660 0x7fde78198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7cb3d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde78112300 con 0x7fde78102240 2024-04-25T12:17:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7cb3d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fde78112180 con 0x7fde78198660 2024-04-25T12:17:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7cb3d640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fde78112480 con 0x7fde78107280 2024-04-25T12:17:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde78102240 0x7fde7819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde78102240 0x7fde7819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47438/0 (socket says 172.21.15.19:47438) 2024-04-25T12:17:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7659c640 1 -- 172.21.15.19:0/2687542924 learned_addr learned my addr 172.21.15.19:0/2687542924 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde76d9d640 1 --2- 172.21.15.19:0/2687542924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde78198660 0x7fde78198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde6ffff640 1 --2- 172.21.15.19:0/2687542924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde78107280 0x7fde7819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7659c640 1 -- 172.21.15.19:0/2687542924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde78107280 msgr2=0x7fde7819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7659c640 1 --2- 172.21.15.19:0/2687542924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde78107280 0x7fde7819eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7659c640 1 -- 172.21.15.19:0/2687542924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde78198660 msgr2=0x7fde78198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7659c640 1 --2- 172.21.15.19:0/2687542924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde78198660 0x7fde78198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.940+0000 7fde7659c640 1 -- 172.21.15.19:0/2687542924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde78199260 con 0x7fde78102240 2024-04-25T12:17:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.941+0000 7fde7659c640 1 --2- 172.21.15.19:0/2687542924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde78102240 0x7fde7819e5a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fde6800c890 tx=0x7fde6800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:53.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.941+0000 7fde6f7fe640 1 -- 172.21.15.19:0/2687542924 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68004220 con 0x7fde78102240 2024-04-25T12:17:53.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.941+0000 7fde6f7fe640 1 -- 172.21.15.19:0/2687542924 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fde680043c0 con 0x7fde78102240 2024-04-25T12:17:53.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.941+0000 7fde7cb3d640 1 -- 172.21.15.19:0/2687542924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde781a5340 con 0x7fde78102240 2024-04-25T12:17:53.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.941+0000 7fde6f7fe640 1 -- 172.21.15.19:0/2687542924 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde680049c0 con 0x7fde78102240 2024-04-25T12:17:53.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.941+0000 7fde7cb3d640 1 -- 172.21.15.19:0/2687542924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde781a56f0 con 0x7fde78102240 2024-04-25T12:17:53.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.942+0000 7fde7cb3d640 1 -- 172.21.15.19:0/2687542924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde78103680 con 0x7fde78102240 2024-04-25T12:17:53.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.943+0000 7fde6f7fe640 1 -- 172.21.15.19:0/2687542924 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fde68005140 con 0x7fde78102240 2024-04-25T12:17:53.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.943+0000 7fde6f7fe640 1 --2- 172.21.15.19:0/2687542924 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fde640a3d20 0x7fde640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:53.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.944+0000 7fde6f7fe640 1 -- 172.21.15.19:0/2687542924 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fde680d0530 con 0x7fde78102240 2024-04-25T12:17:53.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.944+0000 7fde6ffff640 1 --2- 172.21.15.19:0/2687542924 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fde640a3d20 0x7fde640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:53.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.944+0000 7fde6ffff640 1 --2- 172.21.15.19:0/2687542924 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fde640a3d20 0x7fde640a61e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fde60009230 tx=0x7fde60031be0 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:53.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:53.948+0000 7fde6f7fe640 1 -- 172.21.15.19:0/2687542924 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde68014030 con 0x7fde78102240 2024-04-25T12:17:54.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.174+0000 7fde7cb3d640 1 -- 172.21.15.19:0/2687542924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde7810a500 con 0x7fde78102240 2024-04-25T12:17:54.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.175+0000 7fde6f7fe640 1 -- 172.21.15.19:0/2687542924 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7fde6809cb40 con 0x7fde78102240 2024-04-25T12:17:54.176 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:17:54.176 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3292992712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":3292992712},{"type":"v1","addr":"172.21.15.19:6835","nonce":3292992712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/3684868666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":3684868666},{"type":"v1","addr":"172.21.15.19:6837","nonce":3684868666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T12:05:04.008679+0000","modified":"2024-04-25T12:07:07.722007+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24514,"mds_1":24500,"mds_2":24496,"mds_3":24484,"mds_4":14655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14643":{"gid":14643,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/1694897630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1694897630},{"type":"v1","addr":"172.21.15.19:6839","nonce":1694897630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14655":{"gid":14655,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.19:6841/3247157172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3247157172},{"type":"v1","addr":"172.21.15.19:6841","nonce":3247157172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6833/4176941259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":4176941259},{"type":"v1","addr":"172.21.15.150:6833","nonce":4176941259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.169:6833/2217300694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2217300694},{"type":"v1","addr":"172.21.15.169:6833","nonce":2217300694}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/1405540140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1405540140},{"type":"v1","addr":"172.21.15.150:6835","nonce":1405540140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6837/3918801450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3918801450},{"type":"v1","addr":"172.21.15.150:6837","nonce":3918801450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":9,"addr":"172.21.15.169:6835/3316630328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6834","nonce":3316630328},{"type":"v1","addr":"172.21.15.169:6835","nonce":3316630328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.169:6837/1253767842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6836","nonce":1253767842},{"type":"v1","addr":"172.21.15.169:6837","nonce":1253767842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":4,"addr":"172.21.15.150:6839/2553528303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2553528303},{"type":"v1","addr":"172.21.15.150:6839","nonce":2553528303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.169:6839/1723439500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6838","nonce":1723439500},{"type":"v1","addr":"172.21.15.169:6839","nonce":1723439500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[14655,24484,24496,24500,24514]},"id":1}]} 2024-04-25T12:17:54.177 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-04-25T12:17:54.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.177+0000 7fde7cb3d640 1 -- 172.21.15.19:0/2687542924 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fde640a3d20 msgr2=0x7fde640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.177+0000 7fde7cb3d640 1 --2- 172.21.15.19:0/2687542924 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fde640a3d20 0x7fde640a61e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fde60009230 tx=0x7fde60031be0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.177+0000 7fde7cb3d640 1 -- 172.21.15.19:0/2687542924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde78102240 msgr2=0x7fde7819e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.177+0000 7fde7cb3d640 1 --2- 172.21.15.19:0/2687542924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde78102240 0x7fde7819e5a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fde6800c890 tx=0x7fde6800cd50 comp rx=0 tx=0).stop 2024-04-25T12:17:54.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.177+0000 7fde7cb3d640 1 -- 172.21.15.19:0/2687542924 shutdown_connections 2024-04-25T12:17:54.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.177+0000 7fde7cb3d640 1 --2- 172.21.15.19:0/2687542924 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fde640a3d20 0x7fde640a61e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.177+0000 7fde7cb3d640 1 --2- 172.21.15.19:0/2687542924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde78198660 0x7fde78198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.177+0000 7fde7cb3d640 1 --2- 172.21.15.19:0/2687542924 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde78107280 0x7fde7819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.177+0000 7fde7cb3d640 1 --2- 172.21.15.19:0/2687542924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde78102240 0x7fde7819e5a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.177+0000 7fde7cb3d640 1 -- 172.21.15.19:0/2687542924 >> 172.21.15.19:0/2687542924 conn(0x7fde780790e0 msgr2=0x7fde78100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:54.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.177+0000 7fde7cb3d640 1 -- 172.21.15.19:0/2687542924 shutdown_connections 2024-04-25T12:17:54.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.177+0000 7fde7cb3d640 1 -- 172.21.15.19:0/2687542924 wait complete. 2024-04-25T12:17:54.189 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T12:17:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.291+0000 7fe239633640 1 -- 172.21.15.19:0/3295877602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe23406b2b0 msgr2=0x7fe234106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.291+0000 7fe239633640 1 --2- 172.21.15.19:0/3295877602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe23406b2b0 0x7fe234106c20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009900 tx=0x7fe21c02a410 comp rx=0 tx=0).stop 2024-04-25T12:17:54.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.291+0000 7fe239633640 1 -- 172.21.15.19:0/3295877602 shutdown_connections 2024-04-25T12:17:54.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.291+0000 7fe239633640 1 --2- 172.21.15.19:0/3295877602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe234107300 0x7fe23410eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.291+0000 7fe239633640 1 --2- 172.21.15.19:0/3295877602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe23406b2b0 0x7fe234106c20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.291+0000 7fe239633640 1 --2- 172.21.15.19:0/3295877602 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe23406a900 0x7fe23406ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.291+0000 7fe239633640 1 -- 172.21.15.19:0/3295877602 >> 172.21.15.19:0/3295877602 conn(0x7fe234077f20 msgr2=0x7fe234078330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:54.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.291+0000 7fe239633640 1 -- 172.21.15.19:0/3295877602 shutdown_connections 2024-04-25T12:17:54.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.291+0000 7fe239633640 1 -- 172.21.15.19:0/3295877602 wait complete. 2024-04-25T12:17:54.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe239633640 1 Processor -- start 2024-04-25T12:17:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe239633640 1 -- start start 2024-04-25T12:17:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe239633640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe23406a900 0x7fe234100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe239633640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe234107300 0x7fe2341013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe239633640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe234101900 0x7fe2341a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe239633640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe234112300 con 0x7fe234107300 2024-04-25T12:17:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe239633640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe234112180 con 0x7fe234101900 2024-04-25T12:17:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe239633640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe234112480 con 0x7fe23406a900 2024-04-25T12:17:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe22bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe234107300 0x7fe2341013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe22bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe234107300 0x7fe2341013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47460/0 (socket says 172.21.15.19:47460) 2024-04-25T12:17:54.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe22bfff640 1 -- 172.21.15.19:0/1854122018 learned_addr learned my addr 172.21.15.19:0/1854122018 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:54.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe232ffd640 1 --2- 172.21.15.19:0/1854122018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe23406a900 0x7fe234100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:54.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe2337fe640 1 --2- 172.21.15.19:0/1854122018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe234101900 0x7fe2341a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:54.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe22bfff640 1 -- 172.21.15.19:0/1854122018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe23406a900 msgr2=0x7fe234100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe22bfff640 1 --2- 172.21.15.19:0/1854122018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe23406a900 0x7fe234100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.292+0000 7fe22bfff640 1 -- 172.21.15.19:0/1854122018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe234101900 msgr2=0x7fe2341a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.293+0000 7fe22bfff640 1 --2- 172.21.15.19:0/1854122018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe234101900 0x7fe2341a69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.293+0000 7fe22bfff640 1 -- 172.21.15.19:0/1854122018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2341a70d0 con 0x7fe234107300 2024-04-25T12:17:54.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.293+0000 7fe22bfff640 1 --2- 172.21.15.19:0/1854122018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe234107300 0x7fe2341013c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe21c005940 tx=0x7fe21c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:54.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.293+0000 7fe230ff9640 1 -- 172.21.15.19:0/1854122018 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c03f030 con 0x7fe234107300 2024-04-25T12:17:54.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.293+0000 7fe230ff9640 1 -- 172.21.15.19:0/1854122018 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe21c031910 con 0x7fe234107300 2024-04-25T12:17:54.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.293+0000 7fe239633640 1 -- 172.21.15.19:0/1854122018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2341a7360 con 0x7fe234107300 2024-04-25T12:17:54.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.293+0000 7fe230ff9640 1 -- 172.21.15.19:0/1854122018 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c035070 con 0x7fe234107300 2024-04-25T12:17:54.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.293+0000 7fe239633640 1 -- 172.21.15.19:0/1854122018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2341a7850 con 0x7fe234107300 2024-04-25T12:17:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.294+0000 7fe239633640 1 -- 172.21.15.19:0/1854122018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2340691c0 con 0x7fe234107300 2024-04-25T12:17:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.295+0000 7fe230ff9640 1 -- 172.21.15.19:0/1854122018 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe21c042030 con 0x7fe234107300 2024-04-25T12:17:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.296+0000 7fe230ff9640 1 --2- 172.21.15.19:0/1854122018 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fe2200a3d50 0x7fe2200a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.296+0000 7fe232ffd640 1 --2- 172.21.15.19:0/1854122018 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fe2200a3d50 0x7fe2200a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:54.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.296+0000 7fe230ff9640 1 -- 172.21.15.19:0/1854122018 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fe21c0f0fe0 con 0x7fe234107300 2024-04-25T12:17:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.296+0000 7fe232ffd640 1 --2- 172.21.15.19:0/1854122018 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fe2200a3d50 0x7fe2200a6210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe224005fd0 tx=0x7fe224005e20 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:54.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.300+0000 7fe230ff9640 1 -- 172.21.15.19:0/1854122018 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe21c0bd5f0 con 0x7fe234107300 2024-04-25T12:17:54.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.487+0000 7fe239633640 1 -- 172.21.15.19:0/1854122018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2341023b0 con 0x7fe234107300 2024-04-25T12:17:54.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.489+0000 7fe230ff9640 1 -- 172.21.15.19:0/1854122018 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+23199 (secure 0 0 0) 0x7fe2341023b0 con 0x7fe234107300 2024-04-25T12:17:54.490 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-04-25T12:17:54.491 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":96,"fsid":"be62e290-02fa-11ef-bc93-c7b262605968","created":"2024-04-25T11:57:00.082417+0000","modified":"2024-04-25T12:17:15.346984+0000","last_up_change":"2024-04-25T12:04:13.913380+0000","last_in_change":"2024-04-25T12:03:53.656692+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T12:00:20.898809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T12:05:01.658354+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T12:05:02.969078+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73d2b9f6-57f7-4008-9269-75c46243c8e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6803","nonce":86476755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6805","nonce":86476755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6809","nonce":86476755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":86476755},{"type":"v1","addr":"172.21.15.19:6807","nonce":86476755}]},"public_addr":"172.21.15.19:6803/86476755","cluster_addr":"172.21.15.19:6805/86476755","heartbeat_back_addr":"172.21.15.19:6809/86476755","heartbeat_front_addr":"172.21.15.19:6807/86476755","state":["exists","up"]},{"osd":1,"uuid":"9ac46524-3ca7-46f2-9b6e-ca9ce8376ca0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6801","nonce":1593189601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6803","nonce":1593189601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6807","nonce":1593189601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1593189601},{"type":"v1","addr":"172.21.15.150:6805","nonce":1593189601}]},"public_addr":"172.21.15.150:6801/1593189601","cluster_addr":"172.21.15.150:6803/1593189601","heartbeat_back_addr":"172.21.15.150:6807/1593189601","heartbeat_front_addr":"172.21.15.150:6805/1593189601","state":["exists","up"]},{"osd":2,"uuid":"3a12732f-3dd6-4d97-b62a-2bd66a797b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6801","nonce":3641724528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6803","nonce":3641724528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6807","nonce":3641724528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3641724528},{"type":"v1","addr":"172.21.15.169:6805","nonce":3641724528}]},"public_addr":"172.21.15.169:6801/3641724528","cluster_addr":"172.21.15.169:6803/3641724528","heartbeat_back_addr":"172.21.15.169:6807/3641724528","heartbeat_front_addr":"172.21.15.169:6805/3641724528","state":["exists","up"]},{"osd":3,"uuid":"518328cd-b7de-4a49-9045-2ebf6b0eb4ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6811","nonce":3285522913}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6813","nonce":3285522913}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6817","nonce":3285522913}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3285522913},{"type":"v1","addr":"172.21.15.19:6815","nonce":3285522913}]},"public_addr":"172.21.15.19:6811/3285522913","cluster_addr":"172.21.15.19:6813/3285522913","heartbeat_back_addr":"172.21.15.19:6817/3285522913","heartbeat_front_addr":"172.21.15.19:6815/3285522913","state":["exists","up"]},{"osd":4,"uuid":"14ddc9b5-1675-428c-b759-44c255d90bac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6809","nonce":2162522887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6811","nonce":2162522887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6815","nonce":2162522887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2162522887},{"type":"v1","addr":"172.21.15.150:6813","nonce":2162522887}]},"public_addr":"172.21.15.150:6809/2162522887","cluster_addr":"172.21.15.150:6811/2162522887","heartbeat_back_addr":"172.21.15.150:6815/2162522887","heartbeat_front_addr":"172.21.15.150:6813/2162522887","state":["exists","up"]},{"osd":5,"uuid":"99b7ca08-7d93-405d-9f28-9a98b7b3f669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6809","nonce":502865081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6811","nonce":502865081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6815","nonce":502865081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":502865081},{"type":"v1","addr":"172.21.15.169:6813","nonce":502865081}]},"public_addr":"172.21.15.169:6809/502865081","cluster_addr":"172.21.15.169:6811/502865081","heartbeat_back_addr":"172.21.15.169:6815/502865081","heartbeat_front_addr":"172.21.15.169:6813/502865081","state":["exists","up"]},{"osd":6,"uuid":"fb7ed0e1-c6fa-4d4b-88fb-5a04e068bfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6819","nonce":814529084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6821","nonce":814529084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6825","nonce":814529084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":814529084},{"type":"v1","addr":"172.21.15.19:6823","nonce":814529084}]},"public_addr":"172.21.15.19:6819/814529084","cluster_addr":"172.21.15.19:6821/814529084","heartbeat_back_addr":"172.21.15.19:6825/814529084","heartbeat_front_addr":"172.21.15.19:6823/814529084","state":["exists","up"]},{"osd":7,"uuid":"e19b7274-6545-43ee-891a-30f25c2c2e41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6817","nonce":4020549658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6819","nonce":4020549658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6823","nonce":4020549658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":4020549658},{"type":"v1","addr":"172.21.15.150:6821","nonce":4020549658}]},"public_addr":"172.21.15.150:6817/4020549658","cluster_addr":"172.21.15.150:6819/4020549658","heartbeat_back_addr":"172.21.15.150:6823/4020549658","heartbeat_front_addr":"172.21.15.150:6821/4020549658","state":["exists","up"]},{"osd":8,"uuid":"8175ec26-2033-4312-8778-2d2e8512e4cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6817","nonce":3896931254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6819","nonce":3896931254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6823","nonce":3896931254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":3896931254},{"type":"v1","addr":"172.21.15.169:6821","nonce":3896931254}]},"public_addr":"172.21.15.169:6817/3896931254","cluster_addr":"172.21.15.169:6819/3896931254","heartbeat_back_addr":"172.21.15.169:6823/3896931254","heartbeat_front_addr":"172.21.15.169:6821/3896931254","state":["exists","up"]},{"osd":9,"uuid":"2941192e-f2a5-447c-8185-61f976b616db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6827","nonce":3504107664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6829","nonce":3504107664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6833","nonce":3504107664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":3504107664},{"type":"v1","addr":"172.21.15.19:6831","nonce":3504107664}]},"public_addr":"172.21.15.19:6827/3504107664","cluster_addr":"172.21.15.19:6829/3504107664","heartbeat_back_addr":"172.21.15.19:6833/3504107664","heartbeat_front_addr":"172.21.15.19:6831/3504107664","state":["exists","up"]},{"osd":10,"uuid":"e97514b7-bafc-4e07-8afb-3d33ab586ef9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6825","nonce":2984224024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6827","nonce":2984224024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6831","nonce":2984224024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2984224024},{"type":"v1","addr":"172.21.15.150:6829","nonce":2984224024}]},"public_addr":"172.21.15.150:6825/2984224024","cluster_addr":"172.21.15.150:6827/2984224024","heartbeat_back_addr":"172.21.15.150:6831/2984224024","heartbeat_front_addr":"172.21.15.150:6829/2984224024","state":["exists","up"]},{"osd":11,"uuid":"98d4c806-cf93-41d4-9b62-9febe86e9372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6825","nonce":2387365529}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6827","nonce":2387365529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6831","nonce":2387365529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":2387365529},{"type":"v1","addr":"172.21.15.169:6829","nonce":2387365529}]},"public_addr":"172.21.15.169:6825/2387365529","cluster_addr":"172.21.15.169:6827/2387365529","heartbeat_back_addr":"172.21.15.169:6831/2387365529","heartbeat_front_addr":"172.21.15.169:6829/2387365529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:30.716326+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:59:55.364004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:19.383826+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:00:44.505124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:10.255049+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:01:35.227331+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:01.062095+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:27.468735+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:02:52.614495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:19.064870+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:03:45.618387+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T12:04:12.319338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/2813829603":"2024-04-26T12:07:09.072296+0000","172.21.15.19:0/1048024792":"2024-04-26T12:07:09.072296+0000","172.21.15.19:6800/2853466745":"2024-04-26T12:07:09.072296+0000","172.21.15.19:0/1094943085":"2024-04-26T12:07:09.072296+0000","172.21.15.19:0/37557166":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/2734487323":"2024-04-26T12:07:09.072296+0000","172.21.15.19:6801/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6800/963006875":"2024-04-26T11:57:54.811890+0000","172.21.15.19:6801/2853466745":"2024-04-26T12:07:09.072296+0000","172.21.15.19:6800/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:6801/556620573":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1079165232":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/4104025927":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/2968863069":"2024-04-26T11:57:54.811890+0000","172.21.15.19:0/3693313603":"2024-04-26T11:57:27.561732+0000","172.21.15.19:0/1919062753":"2024-04-26T11:57:27.561732+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T12:17:54.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.491+0000 7fe239633640 1 -- 172.21.15.19:0/1854122018 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fe2200a3d50 msgr2=0x7fe2200a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.491+0000 7fe239633640 1 --2- 172.21.15.19:0/1854122018 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fe2200a3d50 0x7fe2200a6210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe224005fd0 tx=0x7fe224005e20 comp rx=0 tx=0).stop 2024-04-25T12:17:54.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.491+0000 7fe239633640 1 -- 172.21.15.19:0/1854122018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe234107300 msgr2=0x7fe2341013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.491+0000 7fe239633640 1 --2- 172.21.15.19:0/1854122018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe234107300 0x7fe2341013c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe21c005940 tx=0x7fe21c0042f0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.491+0000 7fe239633640 1 -- 172.21.15.19:0/1854122018 shutdown_connections 2024-04-25T12:17:54.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.491+0000 7fe239633640 1 --2- 172.21.15.19:0/1854122018 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fe2200a3d50 0x7fe2200a6210 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.491+0000 7fe239633640 1 --2- 172.21.15.19:0/1854122018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe234101900 0x7fe2341a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.491+0000 7fe239633640 1 --2- 172.21.15.19:0/1854122018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe234107300 0x7fe2341013c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.491+0000 7fe239633640 1 --2- 172.21.15.19:0/1854122018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe23406a900 0x7fe234100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.491+0000 7fe239633640 1 -- 172.21.15.19:0/1854122018 >> 172.21.15.19:0/1854122018 conn(0x7fe234077f20 msgr2=0x7fe2340ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:54.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.491+0000 7fe239633640 1 -- 172.21.15.19:0/1854122018 shutdown_connections 2024-04-25T12:17:54.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.491+0000 7fe239633640 1 -- 172.21.15.19:0/1854122018 wait complete. 2024-04-25T12:17:54.502 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:54 smithi019 ceph-mon[29213]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:54.502 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:54 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2269825021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:17:54.502 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:54 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2687542924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:17:54.504 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T12:17:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.606+0000 7f47298fd640 1 -- 172.21.15.19:0/293548033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724102b30 msgr2=0x7f4724106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.606+0000 7f47298fd640 1 --2- 172.21.15.19:0/293548033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724102b30 0x7f4724106b70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4718009900 tx=0x7f471802a410 comp rx=0 tx=0).stop 2024-04-25T12:17:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.606+0000 7f47298fd640 1 -- 172.21.15.19:0/293548033 shutdown_connections 2024-04-25T12:17:54.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.606+0000 7f47298fd640 1 --2- 172.21.15.19:0/293548033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4724107250 0x7f472410ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.606+0000 7f47298fd640 1 --2- 172.21.15.19:0/293548033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724102b30 0x7f4724106b70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.606+0000 7f47298fd640 1 --2- 172.21.15.19:0/293548033 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4724102180 0x7f4724102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.606+0000 7f47298fd640 1 -- 172.21.15.19:0/293548033 >> 172.21.15.19:0/293548033 conn(0x7f47240792d0 msgr2=0x7f47240796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.606+0000 7f47298fd640 1 -- 172.21.15.19:0/293548033 shutdown_connections 2024-04-25T12:17:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.606+0000 7f47298fd640 1 -- 172.21.15.19:0/293548033 wait complete. 2024-04-25T12:17:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.606+0000 7f47298fd640 1 Processor -- start 2024-04-25T12:17:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47298fd640 1 -- start start 2024-04-25T12:17:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47298fd640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4724102180 0x7f472419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47298fd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4724107250 0x7f472419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47298fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724198670 0x7f4724198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47298fd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4724112330 con 0x7f4724198670 2024-04-25T12:17:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47298fd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f47241121b0 con 0x7f4724107250 2024-04-25T12:17:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47298fd640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f47241124b0 con 0x7f4724102180 2024-04-25T12:17:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724198670 0x7f4724198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724198670 0x7f4724198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47470/0 (socket says 172.21.15.19:47470) 2024-04-25T12:17:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47237fe640 1 -- 172.21.15.19:0/561596860 learned_addr learned my addr 172.21.15.19:0/561596860 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47227fc640 1 --2- 172.21.15.19:0/561596860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4724107250 0x7f472419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f4722ffd640 1 --2- 172.21.15.19:0/561596860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4724102180 0x7f472419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47237fe640 1 -- 172.21.15.19:0/561596860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4724102180 msgr2=0x7f472419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47237fe640 1 --2- 172.21.15.19:0/561596860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4724102180 0x7f472419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47237fe640 1 -- 172.21.15.19:0/561596860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4724107250 msgr2=0x7f472419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47237fe640 1 --2- 172.21.15.19:0/561596860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4724107250 0x7f472419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47237fe640 1 -- 172.21.15.19:0/561596860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4724199270 con 0x7f4724198670 2024-04-25T12:17:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.607+0000 7f47237fe640 1 --2- 172.21.15.19:0/561596860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724198670 0x7f4724198af0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f471c00cad0 tx=0x7f471c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.608+0000 7f47288fb640 1 -- 172.21.15.19:0/561596860 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c00ae20 con 0x7f4724198670 2024-04-25T12:17:54.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.608+0000 7f47298fd640 1 -- 172.21.15.19:0/561596860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47241a5350 con 0x7f4724198670 2024-04-25T12:17:54.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.608+0000 7f47288fb640 1 -- 172.21.15.19:0/561596860 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f471c005020 con 0x7f4724198670 2024-04-25T12:17:54.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.608+0000 7f47288fb640 1 -- 172.21.15.19:0/561596860 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c00f6b0 con 0x7f4724198670 2024-04-25T12:17:54.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.608+0000 7f47298fd640 1 -- 172.21.15.19:0/561596860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47241a5700 con 0x7f4724198670 2024-04-25T12:17:54.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.609+0000 7f47298fd640 1 -- 172.21.15.19:0/561596860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4724103650 con 0x7f4724198670 2024-04-25T12:17:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.610+0000 7f47288fb640 1 -- 172.21.15.19:0/561596860 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f471c0051d0 con 0x7f4724198670 2024-04-25T12:17:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.610+0000 7f47288fb640 1 --2- 172.21.15.19:0/561596860 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f470c0a3d20 0x7f470c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.611+0000 7f4722ffd640 1 --2- 172.21.15.19:0/561596860 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f470c0a3d20 0x7f470c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.611+0000 7f47288fb640 1 -- 172.21.15.19:0/561596860 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f471c0d0920 con 0x7f4724198670 2024-04-25T12:17:54.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.611+0000 7f4722ffd640 1 --2- 172.21.15.19:0/561596860 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f470c0a3d20 0x7f470c0a61e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4714009890 tx=0x7f4714008040 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.615+0000 7f47288fb640 1 -- 172.21.15.19:0/561596860 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f471c014030 con 0x7f4724198670 2024-04-25T12:17:54.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-mon[32710]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:54.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2269825021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:17:54.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2687542924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:17:54.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-mon[32250]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T12:17:54.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2269825021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:17:54.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2687542924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T12:17:54.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.839+0000 7f47298fd640 1 -- 172.21.15.19:0/561596860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4724102b30 con 0x7f4724198670 2024-04-25T12:17:54.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.841+0000 7f47288fb640 1 -- 172.21.15.19:0/561596860 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f471c09cf90 con 0x7f4724198670 2024-04-25T12:17:54.842 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T12:17:54.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.845+0000 7f47298fd640 1 -- 172.21.15.19:0/561596860 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f470c0a3d20 msgr2=0x7f470c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.845+0000 7f47298fd640 1 --2- 172.21.15.19:0/561596860 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f470c0a3d20 0x7f470c0a61e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4714009890 tx=0x7f4714008040 comp rx=0 tx=0).stop 2024-04-25T12:17:54.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.845+0000 7f47298fd640 1 -- 172.21.15.19:0/561596860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724198670 msgr2=0x7f4724198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.845+0000 7f47298fd640 1 --2- 172.21.15.19:0/561596860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724198670 0x7f4724198af0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f471c00cad0 tx=0x7f471c00cf90 comp rx=0 tx=0).stop 2024-04-25T12:17:54.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.845+0000 7f47298fd640 1 -- 172.21.15.19:0/561596860 shutdown_connections 2024-04-25T12:17:54.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.845+0000 7f47298fd640 1 --2- 172.21.15.19:0/561596860 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f470c0a3d20 0x7f470c0a61e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.845+0000 7f47298fd640 1 --2- 172.21.15.19:0/561596860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724198670 0x7f4724198af0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.845+0000 7f47298fd640 1 --2- 172.21.15.19:0/561596860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4724107250 0x7f472419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.845+0000 7f47298fd640 1 --2- 172.21.15.19:0/561596860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4724102180 0x7f472419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.845+0000 7f47298fd640 1 -- 172.21.15.19:0/561596860 >> 172.21.15.19:0/561596860 conn(0x7f47240792d0 msgr2=0x7f4724100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.845+0000 7f47298fd640 1 -- 172.21.15.19:0/561596860 shutdown_connections 2024-04-25T12:17:54.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.845+0000 7f47298fd640 1 -- 172.21.15.19:0/561596860 wait complete. 2024-04-25T12:17:54.858 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T12:17:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.969+0000 7f2c9469a640 1 -- 172.21.15.19:0/3199204822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8c107270 msgr2=0x7f2c8c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.969+0000 7f2c9469a640 1 --2- 172.21.15.19:0/3199204822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8c107270 0x7f2c8c10ee40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009ca0 tx=0x7f2c8402a8c0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.969+0000 7f2c9469a640 1 -- 172.21.15.19:0/3199204822 shutdown_connections 2024-04-25T12:17:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.969+0000 7f2c9469a640 1 --2- 172.21.15.19:0/3199204822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8c107270 0x7f2c8c10ee40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.969+0000 7f2c9469a640 1 --2- 172.21.15.19:0/3199204822 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c8c102b50 0x7f2c8c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.969+0000 7f2c9469a640 1 --2- 172.21.15.19:0/3199204822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c8c1021a0 0x7f2c8c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.969+0000 7f2c9469a640 1 -- 172.21.15.19:0/3199204822 >> 172.21.15.19:0/3199204822 conn(0x7f2c8c079350 msgr2=0x7f2c8c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.969+0000 7f2c9469a640 1 -- 172.21.15.19:0/3199204822 shutdown_connections 2024-04-25T12:17:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.969+0000 7f2c9469a640 1 -- 172.21.15.19:0/3199204822 wait complete. 2024-04-25T12:17:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.969+0000 7f2c9469a640 1 Processor -- start 2024-04-25T12:17:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.969+0000 7f2c9469a640 1 -- start start 2024-04-25T12:17:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c9469a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8c1021a0 0x7f2c8c19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c9469a640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c8c107270 0x7f2c8c19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c9469a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c8c198650 0x7f2c8c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c9469a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c112310 con 0x7f2c8c1021a0 2024-04-25T12:17:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c9469a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c112190 con 0x7f2c8c198650 2024-04-25T12:17:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c9469a640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c112490 con 0x7f2c8c107270 2024-04-25T12:17:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c92436640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8c1021a0 0x7f2c8c19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c92436640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8c1021a0 0x7f2c8c19e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47508/0 (socket says 172.21.15.19:47508) 2024-04-25T12:17:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c92436640 1 -- 172.21.15.19:0/2064019752 learned_addr learned my addr 172.21.15.19:0/2064019752 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c92c37640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c8c198650 0x7f2c8c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c91c35640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c8c107270 0x7f2c8c19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c92436640 1 -- 172.21.15.19:0/2064019752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c8c107270 msgr2=0x7f2c8c19ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c92436640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c8c107270 0x7f2c8c19ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c92436640 1 -- 172.21.15.19:0/2064019752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c8c198650 msgr2=0x7f2c8c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c92436640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c8c198650 0x7f2c8c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c92436640 1 -- 172.21.15.19:0/2064019752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c199250 con 0x7f2c8c1021a0 2024-04-25T12:17:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c92c37640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c8c198650 0x7f2c8c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:17:54.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c92436640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8c1021a0 0x7f2c8c19e590 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800ea60 tx=0x7f2c8800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:54.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c837fe640 1 -- 172.21.15.19:0/2064019752 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8800cde0 con 0x7f2c8c1021a0 2024-04-25T12:17:54.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c837fe640 1 -- 172.21.15.19:0/2064019752 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f2c880035f0 con 0x7f2c8c1021a0 2024-04-25T12:17:54.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c9469a640 1 -- 172.21.15.19:0/2064019752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8c1a5330 con 0x7f2c8c1021a0 2024-04-25T12:17:54.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c837fe640 1 -- 172.21.15.19:0/2064019752 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88010690 con 0x7f2c8c1021a0 2024-04-25T12:17:54.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.970+0000 7f2c9469a640 1 -- 172.21.15.19:0/2064019752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8c1a5730 con 0x7f2c8c1021a0 2024-04-25T12:17:54.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.971+0000 7f2c9469a640 1 -- 172.21.15.19:0/2064019752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c8c103670 con 0x7f2c8c1021a0 2024-04-25T12:17:54.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.973+0000 7f2c837fe640 1 -- 172.21.15.19:0/2064019752 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c880037a0 con 0x7f2c8c1021a0 2024-04-25T12:17:54.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.973+0000 7f2c837fe640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f2c780a3d20 0x7f2c780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:54.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.973+0000 7f2c91c35640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f2c780a3d20 0x7f2c780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:54.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.973+0000 7f2c837fe640 1 -- 172.21.15.19:0/2064019752 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8957+0+0 (secure 0 0 0) 0x7f2c880d1060 con 0x7f2c8c1021a0 2024-04-25T12:17:54.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.974+0000 7f2c91c35640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f2c780a3d20 0x7f2c780a61e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c199ea0 tx=0x7f2c7c005f90 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:54.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:54.977+0000 7f2c837fe640 1 -- 172.21.15.19:0/2064019752 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c88053ff0 con 0x7f2c8c1021a0 2024-04-25T12:17:55.159 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-c[64164]: ignoring --setuser ceph since I am not root 2024-04-25T12:17:55.159 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-c[64164]: ignoring --setgroup ceph since I am not root 2024-04-25T12:17:55.159 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-mds[64168]: -- 172.21.15.169:0/1747527503 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5565b55a24e0 con 0x5565b5578800 2024-04-25T12:17:55.159 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-mds[64168]: -- 172.21.15.169:0/1747527503 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565b55a5a40 con 0x5565b5578800 2024-04-25T12:17:55.160 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-c[64164]: starting mds.c at 2024-04-25T12:17:55.160 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-f[63859]: ignoring --setuser ceph since I am not root 2024-04-25T12:17:55.160 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-f[63859]: ignoring --setgroup ceph since I am not root 2024-04-25T12:17:55.160 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-mds[63863]: -- 172.21.15.169:0/254828353 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5558eb617a40 con 0x5558eb5eac00 2024-04-25T12:17:55.160 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-f[63859]: starting mds.f at 2024-04-25T12:17:55.161 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-l[63541]: ignoring --setuser ceph since I am not root 2024-04-25T12:17:55.161 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-l[63541]: ignoring --setgroup ceph since I am not root 2024-04-25T12:17:55.161 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-mds[63545]: -- 172.21.15.169:0/24643429 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608e3619a40 con 0x5608e35ed000 2024-04-25T12:17:55.161 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-l[63541]: starting mds.l at 2024-04-25T12:17:55.161 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i[64470]: ignoring --setuser ceph since I am not root 2024-04-25T12:17:55.161 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i[64470]: ignoring --setgroup ceph since I am not root 2024-04-25T12:17:55.162 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-mds[64474]: -- 172.21.15.169:0/499786719 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb2501ba40 con 0x55bb24feec00 2024-04-25T12:17:55.162 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:17:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i[64470]: starting mds.i at 2024-04-25T12:17:55.166 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-k[61890]: ignoring --setuser ceph since I am not root 2024-04-25T12:17:55.166 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-k[61890]: ignoring --setgroup ceph since I am not root 2024-04-25T12:17:55.167 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-mds[61894]: -- 172.21.15.150:0/3899269226 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d194745a40 con 0x55d194718800 2024-04-25T12:17:55.167 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-k[61890]: starting mds.k at 2024-04-25T12:17:55.167 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-k[61890]: 2024-04-25T12:17:54.890+0000 7f30452de700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T12:17:55.167 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-e[62211]: ignoring --setuser ceph since I am not root 2024-04-25T12:17:55.167 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-e[62211]: ignoring --setgroup ceph since I am not root 2024-04-25T12:17:55.167 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-e[62211]: starting mds.e at 2024-04-25T12:17:55.168 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-h[62521]: ignoring --setuser ceph since I am not root 2024-04-25T12:17:55.168 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-h[62521]: ignoring --setgroup ceph since I am not root 2024-04-25T12:17:55.168 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-mds[62525]: -- 172.21.15.150:0/2791674584 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561de1dcfa40 con 0x561de1da3000 2024-04-25T12:17:55.168 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:17:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-h[62521]: starting mds.h at 2024-04-25T12:17:55.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.202+0000 7f2c9469a640 1 -- 172.21.15.19:0/2064019752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2c8c10a560 con 0x7f2c8c1021a0 2024-04-25T12:17:55.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a[29209]: 2024-04-25T12:17:54.839+0000 7fe736d72700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T12:17:55.301 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:17:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-g[71385]: ignoring --setuser ceph since I am not root 2024-04-25T12:17:55.301 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:17:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-g[71385]: ignoring --setgroup ceph since I am not root 2024-04-25T12:17:55.301 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:17:54 smithi019 ceph-mds[71389]: -- 172.21.15.19:0/3249610612 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561b2fb3fa40 con 0x561b2fb13000 2024-04-25T12:17:55.301 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:17:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-g[71385]: starting mds.g at 2024-04-25T12:17:55.302 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:17:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-a[71737]: ignoring --setuser ceph since I am not root 2024-04-25T12:17:55.302 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:17:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-a[71737]: ignoring --setgroup ceph since I am not root 2024-04-25T12:17:55.302 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:17:54 smithi019 ceph-mds[71741]: -- 172.21.15.19:0/3049448316 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b25cb5a40 con 0x558b25c88800 2024-04-25T12:17:55.303 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:17:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-a[71737]: starting mds.a at 2024-04-25T12:17:55.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:55 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/1854122018' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:17:55.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:55 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/561596860' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T12:17:55.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:55 smithi169 ceph-mon[32710]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T12:17:55.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:55 smithi169 ceph-mon[32710]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T12:17:55.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:55 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/561596860' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T12:17:55.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:55 smithi169 ceph-mon[32710]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T12:17:55.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:55 smithi169 ceph-mon[32710]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T12:17:55.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:55 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2064019752' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:17:55.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:55 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/1854122018' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:17:55.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:55 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/561596860' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T12:17:55.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:55 smithi150 ceph-mon[32250]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T12:17:55.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:55 smithi150 ceph-mon[32250]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T12:17:55.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:55 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/561596860' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T12:17:55.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:55 smithi150 ceph-mon[32250]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T12:17:55.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:55 smithi150 ceph-mon[32250]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T12:17:55.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:55 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2064019752' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:17:55.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:55 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/1854122018' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T12:17:55.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:55 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/561596860' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T12:17:55.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:55 smithi019 ceph-mon[29213]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T12:17:55.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:55 smithi019 ceph-mon[29213]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T12:17:55.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:55 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/561596860' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T12:17:55.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:55 smithi019 ceph-mon[29213]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T12:17:55.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:55 smithi019 ceph-mon[29213]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T12:17:55.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:55 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2064019752' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T12:17:55.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.849+0000 7f2c837fe640 1 -- 172.21.15.19:0/2064019752 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f2c88016330 con 0x7f2c8c1021a0 2024-04-25T12:17:55.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.852+0000 7f2c9469a640 1 -- 172.21.15.19:0/2064019752 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f2c780a3d20 msgr2=0x7f2c780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:55.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.852+0000 7f2c9469a640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f2c780a3d20 0x7f2c780a61e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c199ea0 tx=0x7f2c7c005f90 comp rx=0 tx=0).stop 2024-04-25T12:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.852+0000 7f2c9469a640 1 -- 172.21.15.19:0/2064019752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8c1021a0 msgr2=0x7f2c8c19e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.852+0000 7f2c9469a640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8c1021a0 0x7f2c8c19e590 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800ea60 tx=0x7f2c8800ef20 comp rx=0 tx=0).stop 2024-04-25T12:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.852+0000 7f2c9469a640 1 -- 172.21.15.19:0/2064019752 shutdown_connections 2024-04-25T12:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.852+0000 7f2c9469a640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f2c780a3d20 0x7f2c780a61e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.852+0000 7f2c9469a640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c8c198650 0x7f2c8c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.852+0000 7f2c9469a640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2c8c107270 0x7f2c8c19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:55.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.852+0000 7f2c9469a640 1 --2- 172.21.15.19:0/2064019752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c8c1021a0 0x7f2c8c19e590 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:55.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.852+0000 7f2c9469a640 1 -- 172.21.15.19:0/2064019752 >> 172.21.15.19:0/2064019752 conn(0x7f2c8c079350 msgr2=0x7f2c8c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:55.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.853+0000 7f2c9469a640 1 -- 172.21.15.19:0/2064019752 shutdown_connections 2024-04-25T12:17:55.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.853+0000 7f2c9469a640 1 -- 172.21.15.19:0/2064019752 wait complete. 2024-04-25T12:17:55.871 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T12:17:55.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.984+0000 7f085d6a0640 1 -- 172.21.15.19:0/2191287638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0858103b00 msgr2=0x7f085810bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:55.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.984+0000 7f085d6a0640 1 --2- 172.21.15.19:0/2191287638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0858103b00 0x7f085810bfe0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0840009900 tx=0x7f084002a510 comp rx=0 tx=0).stop 2024-04-25T12:17:55.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.984+0000 7f085d6a0640 1 -- 172.21.15.19:0/2191287638 shutdown_connections 2024-04-25T12:17:55.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.984+0000 7f085d6a0640 1 --2- 172.21.15.19:0/2191287638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f085810c520 0x7f085810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:55.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.984+0000 7f085d6a0640 1 --2- 172.21.15.19:0/2191287638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0858103b00 0x7f085810bfe0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:55.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.984+0000 7f085d6a0640 1 --2- 172.21.15.19:0/2191287638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08581031e0 0x7f08581035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:55.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.984+0000 7f085d6a0640 1 -- 172.21.15.19:0/2191287638 >> 172.21.15.19:0/2191287638 conn(0x7f08580771d0 msgr2=0x7f08580775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:55.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.984+0000 7f085d6a0640 1 -- 172.21.15.19:0/2191287638 shutdown_connections 2024-04-25T12:17:55.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.984+0000 7f085d6a0640 1 -- 172.21.15.19:0/2191287638 wait complete. 2024-04-25T12:17:55.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.985+0000 7f085d6a0640 1 Processor -- start 2024-04-25T12:17:55.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.985+0000 7f085d6a0640 1 -- start start 2024-04-25T12:17:55.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.985+0000 7f085d6a0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581031e0 0x7f0858198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:55.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.985+0000 7f085d6a0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0858103b00 0x7f0858198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:55.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.985+0000 7f085d6a0640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f085810c520 0x7f085819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:55.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.985+0000 7f085d6a0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0858112340 con 0x7f08581031e0 2024-04-25T12:17:55.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.985+0000 7f085d6a0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f08581121c0 con 0x7f0858103b00 2024-04-25T12:17:55.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.985+0000 7f085d6a0640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7f08581124c0 con 0x7f085810c520 2024-04-25T12:17:55.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.985+0000 7f0856ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581031e0 0x7f0858198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:55.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.985+0000 7f0856ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581031e0 0x7f0858198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47530/0 (socket says 172.21.15.19:47530) 2024-04-25T12:17:55.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.985+0000 7f0856ffd640 1 -- 172.21.15.19:0/2479288829 learned_addr learned my addr 172.21.15.19:0/2479288829 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:55.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.985+0000 7f08577fe640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f085810c520 0x7f085819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:55.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.985+0000 7f084e7fc640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0858103b00 0x7f0858198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:55.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.986+0000 7f0856ffd640 1 -- 172.21.15.19:0/2479288829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f085810c520 msgr2=0x7f085819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:55.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.986+0000 7f0856ffd640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f085810c520 0x7f085819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:55.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.986+0000 7f0856ffd640 1 -- 172.21.15.19:0/2479288829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0858103b00 msgr2=0x7f0858198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:55.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.986+0000 7f0856ffd640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0858103b00 0x7f0858198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:55.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.986+0000 7f0856ffd640 1 -- 172.21.15.19:0/2479288829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085819d750 con 0x7f08581031e0 2024-04-25T12:17:55.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.986+0000 7f08577fe640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f085810c520 0x7f085819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T12:17:55.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.986+0000 7f0856ffd640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581031e0 0x7f0858198760 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f084800c920 tx=0x7f084800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:55.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.986+0000 7f0854ff9640 1 -- 172.21.15.19:0/2479288829 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848007c70 con 0x7f08581031e0 2024-04-25T12:17:55.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.986+0000 7f085d6a0640 1 -- 172.21.15.19:0/2479288829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085819da40 con 0x7f08581031e0 2024-04-25T12:17:55.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.986+0000 7f0854ff9640 1 -- 172.21.15.19:0/2479288829 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7f0848007e10 con 0x7f08581031e0 2024-04-25T12:17:55.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.986+0000 7f0854ff9640 1 -- 172.21.15.19:0/2479288829 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08480188c0 con 0x7f08581031e0 2024-04-25T12:17:55.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.986+0000 7f085d6a0640 1 -- 172.21.15.19:0/2479288829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08581a52d0 con 0x7f08581031e0 2024-04-25T12:17:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.987+0000 7f085d6a0640 1 -- 172.21.15.19:0/2479288829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08580691c0 con 0x7f08581031e0 2024-04-25T12:17:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.988+0000 7f0854ff9640 1 -- 172.21.15.19:0/2479288829 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0848018a60 con 0x7f08581031e0 2024-04-25T12:17:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.989+0000 7f0854ff9640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f08440a3d20 0x7f08440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.989+0000 7f084e7fc640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f08440a3d20 0x7f08440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.989+0000 7f0854ff9640 1 -- 172.21.15.19:0/2479288829 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8957+0+0 (secure 0 0 0) 0x7f08480d0e60 con 0x7f08581031e0 2024-04-25T12:17:55.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.989+0000 7f084e7fc640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f08440a3d20 0x7f08440a61e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0840000c00 tx=0x7f0840005a20 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:55.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:55.992+0000 7f0854ff9640 1 -- 172.21.15.19:0/2479288829 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0848014030 con 0x7f08581031e0 2024-04-25T12:17:56.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.178+0000 7f085d6a0640 1 -- 172.21.15.19:0/2479288829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0858199ab0 con 0x7f08581031e0 2024-04-25T12:17:56.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 716 B/s wr, 10 op/s 2024-04-25T12:17:56.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T12:17:56.659 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T12:17:56.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: Cluster is now healthy 2024-04-25T12:17:56.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2064019752' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T12:17:56.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.19:6838/1202738827,v1:172.21.15.19:6841/1202738827] up:boot 2024-04-25T12:17:56.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.19:6839/83846878,v1:172.21.15.19:6840/83846878] up:boot 2024-04-25T12:17:56.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.169:6832/34197282,v1:172.21.15.169:6834/34197282] up:boot 2024-04-25T12:17:56.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.169:6833/3468007989,v1:172.21.15.169:6835/3468007989] up:boot 2024-04-25T12:17:56.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.150:6832/3821334610,v1:172.21.15.150:6836/3821334610] up:boot 2024-04-25T12:17:56.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.150:6834/2518838384,v1:172.21.15.150:6837/2518838384] up:boot 2024-04-25T12:17:56.660 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.150:6833/4133506714,v1:172.21.15.150:6838/4133506714] up:boot 2024-04-25T12:17:56.661 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.150:6835/1905552570,v1:172.21.15.150:6839/1905552570] up:boot 2024-04-25T12:17:56.661 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.169:6836/3287928283,v1:172.21.15.169:6837/3287928283] up:boot 2024-04-25T12:17:56.661 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: mds.? [v2:172.21.15.169:6838/2709660558,v1:172.21.15.169:6839/2709660558] up:boot 2024-04-25T12:17:56.661 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: fsmap 12 up:standby 2024-04-25T12:17:56.661 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T12:17:56.661 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T12:17:56.661 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T12:17:56.662 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T12:17:56.662 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T12:17:56.662 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T12:17:56.663 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T12:17:56.663 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T12:17:56.663 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T12:17:56.663 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T12:17:56.663 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:56 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2479288829' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T12:17:56.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 716 B/s wr, 10 op/s 2024-04-25T12:17:56.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T12:17:56.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T12:17:56.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: Cluster is now healthy 2024-04-25T12:17:56.668 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2064019752' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T12:17:56.668 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.19:6838/1202738827,v1:172.21.15.19:6841/1202738827] up:boot 2024-04-25T12:17:56.668 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.19:6839/83846878,v1:172.21.15.19:6840/83846878] up:boot 2024-04-25T12:17:56.669 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.169:6832/34197282,v1:172.21.15.169:6834/34197282] up:boot 2024-04-25T12:17:56.669 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.169:6833/3468007989,v1:172.21.15.169:6835/3468007989] up:boot 2024-04-25T12:17:56.669 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.150:6832/3821334610,v1:172.21.15.150:6836/3821334610] up:boot 2024-04-25T12:17:56.669 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.150:6834/2518838384,v1:172.21.15.150:6837/2518838384] up:boot 2024-04-25T12:17:56.670 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.150:6833/4133506714,v1:172.21.15.150:6838/4133506714] up:boot 2024-04-25T12:17:56.670 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.150:6835/1905552570,v1:172.21.15.150:6839/1905552570] up:boot 2024-04-25T12:17:56.670 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.169:6836/3287928283,v1:172.21.15.169:6837/3287928283] up:boot 2024-04-25T12:17:56.670 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: mds.? [v2:172.21.15.169:6838/2709660558,v1:172.21.15.169:6839/2709660558] up:boot 2024-04-25T12:17:56.671 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: fsmap 12 up:standby 2024-04-25T12:17:56.671 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T12:17:56.671 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T12:17:56.671 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T12:17:56.671 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T12:17:56.671 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T12:17:56.671 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T12:17:56.671 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T12:17:56.672 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T12:17:56.672 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T12:17:56.672 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T12:17:56.672 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:56 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2479288829' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T12:17:56.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 716 B/s wr, 10 op/s 2024-04-25T12:17:56.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T12:17:56.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T12:17:56.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: Cluster is now healthy 2024-04-25T12:17:56.801 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2064019752' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T12:17:56.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.19:6838/1202738827,v1:172.21.15.19:6841/1202738827] up:boot 2024-04-25T12:17:56.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.19:6839/83846878,v1:172.21.15.19:6840/83846878] up:boot 2024-04-25T12:17:56.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.169:6832/34197282,v1:172.21.15.169:6834/34197282] up:boot 2024-04-25T12:17:56.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.169:6833/3468007989,v1:172.21.15.169:6835/3468007989] up:boot 2024-04-25T12:17:56.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.150:6832/3821334610,v1:172.21.15.150:6836/3821334610] up:boot 2024-04-25T12:17:56.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.150:6834/2518838384,v1:172.21.15.150:6837/2518838384] up:boot 2024-04-25T12:17:56.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.150:6833/4133506714,v1:172.21.15.150:6838/4133506714] up:boot 2024-04-25T12:17:56.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.150:6835/1905552570,v1:172.21.15.150:6839/1905552570] up:boot 2024-04-25T12:17:56.803 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.169:6836/3287928283,v1:172.21.15.169:6837/3287928283] up:boot 2024-04-25T12:17:56.803 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: mds.? [v2:172.21.15.169:6838/2709660558,v1:172.21.15.169:6839/2709660558] up:boot 2024-04-25T12:17:56.803 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: fsmap 12 up:standby 2024-04-25T12:17:56.803 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T12:17:56.803 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T12:17:56.803 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T12:17:56.803 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T12:17:56.803 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T12:17:56.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T12:17:56.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T12:17:56.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T12:17:56.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T12:17:56.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: from='mgr.15381 172.21.15.19:0/51072066' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T12:17:56.804 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:56 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2479288829' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T12:17:56.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.851+0000 7f0854ff9640 1 -- 172.21.15.19:0/2479288829 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) ==== 181+0+0 (secure 0 0 0) 0x7f084809d3e0 con 0x7f08581031e0 2024-04-25T12:17:56.853 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-04-25T12:17:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.855+0000 7f085d6a0640 1 -- 172.21.15.19:0/2479288829 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f08440a3d20 msgr2=0x7f08440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:56.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.855+0000 7f085d6a0640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f08440a3d20 0x7f08440a61e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0840000c00 tx=0x7f0840005a20 comp rx=0 tx=0).stop 2024-04-25T12:17:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.855+0000 7f085d6a0640 1 -- 172.21.15.19:0/2479288829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581031e0 msgr2=0x7f0858198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.855+0000 7f085d6a0640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581031e0 0x7f0858198760 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f084800c920 tx=0x7f084800cde0 comp rx=0 tx=0).stop 2024-04-25T12:17:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.855+0000 7f085d6a0640 1 -- 172.21.15.19:0/2479288829 shutdown_connections 2024-04-25T12:17:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.855+0000 7f085d6a0640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7f08440a3d20 0x7f08440a61e0 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0840000c00 tx=0x7f0840005a20 comp rx=0 tx=0).stop 2024-04-25T12:17:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.855+0000 7f085d6a0640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f085810c520 0x7f085819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.855+0000 7f085d6a0640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0858103b00 0x7f0858198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.855+0000 7f085d6a0640 1 --2- 172.21.15.19:0/2479288829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08581031e0 0x7f0858198760 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:56.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.855+0000 7f085d6a0640 1 -- 172.21.15.19:0/2479288829 >> 172.21.15.19:0/2479288829 conn(0x7f08580771d0 msgr2=0x7f08580fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.856+0000 7f085d6a0640 1 -- 172.21.15.19:0/2479288829 shutdown_connections 2024-04-25T12:17:56.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.857+0000 7f085d6a0640 1 -- 172.21.15.19:0/2479288829 wait complete. 2024-04-25T12:17:56.872 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T12:17:56.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.986+0000 7fe7bd272640 1 -- 172.21.15.19:0/3479660458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7b806a840 msgr2=0x7fe7b806ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:56.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.986+0000 7fe7bd272640 1 --2- 172.21.15.19:0/3479660458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7b806a840 0x7fe7b806ac20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009900 tx=0x7fe7a802a410 comp rx=0 tx=0).stop 2024-04-25T12:17:56.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.986+0000 7fe7bd272640 1 -- 172.21.15.19:0/3479660458 shutdown_connections 2024-04-25T12:17:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.986+0000 7fe7bd272640 1 --2- 172.21.15.19:0/3479660458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b8107270 0x7fe7b810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.986+0000 7fe7bd272640 1 --2- 172.21.15.19:0/3479660458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7b806b1f0 0x7fe7b8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.986+0000 7fe7bd272640 1 --2- 172.21.15.19:0/3479660458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7b806a840 0x7fe7b806ac20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.986+0000 7fe7bd272640 1 -- 172.21.15.19:0/3479660458 >> 172.21.15.19:0/3479660458 conn(0x7fe7b8077eb0 msgr2=0x7fe7b80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.986+0000 7fe7bd272640 1 -- 172.21.15.19:0/3479660458 shutdown_connections 2024-04-25T12:17:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.986+0000 7fe7bd272640 1 -- 172.21.15.19:0/3479660458 wait complete. 2024-04-25T12:17:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.987+0000 7fe7bd272640 1 Processor -- start 2024-04-25T12:17:56.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.987+0000 7fe7bd272640 1 -- start start 2024-04-25T12:17:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.987+0000 7fe7bd272640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7b806a840 0x7fe7b819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.987+0000 7fe7bd272640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7b8107270 0x7fe7b819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.987+0000 7fe7bd272640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b81985c0 0x7fe7b8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.987+0000 7fe7bd272640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8112360 con 0x7fe7b8107270 2024-04-25T12:17:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.987+0000 7fe7bd272640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81121e0 con 0x7fe7b81985c0 2024-04-25T12:17:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.987+0000 7fe7bd272640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81124e0 con 0x7fe7b806a840 2024-04-25T12:17:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.987+0000 7fe7affff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7b8107270 0x7fe7b819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.987+0000 7fe7affff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7b8107270 0x7fe7b819e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47560/0 (socket says 172.21.15.19:47560) 2024-04-25T12:17:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.987+0000 7fe7affff640 1 -- 172.21.15.19:0/2640526403 learned_addr learned my addr 172.21.15.19:0/2640526403 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-04-25T12:17:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.987+0000 7fe7b6ffd640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7b806a840 0x7fe7b819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.987+0000 7fe7b77fe640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b81985c0 0x7fe7b8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.988+0000 7fe7affff640 1 -- 172.21.15.19:0/2640526403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7b806a840 msgr2=0x7fe7b819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.988+0000 7fe7affff640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7b806a840 0x7fe7b819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.988+0000 7fe7affff640 1 -- 172.21.15.19:0/2640526403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b81985c0 msgr2=0x7fe7b8198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.988+0000 7fe7affff640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b81985c0 0x7fe7b8198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.988+0000 7fe7affff640 1 -- 172.21.15.19:0/2640526403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b81990e0 con 0x7fe7b8107270 2024-04-25T12:17:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.988+0000 7fe7b77fe640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b81985c0 0x7fe7b8198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T12:17:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.988+0000 7fe7affff640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7b8107270 0x7fe7b819e950 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000d740 tx=0x7fe7a000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.988+0000 7fe7b4ff9640 1 -- 172.21.15.19:0/2640526403 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0017070 con 0x7fe7b8107270 2024-04-25T12:17:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.988+0000 7fe7b4ff9640 1 -- 172.21.15.19:0/2640526403 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1135+0+0 (secure 0 0 0) 0x7fe7a000de50 con 0x7fe7b8107270 2024-04-25T12:17:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.988+0000 7fe7bd272640 1 -- 172.21.15.19:0/2640526403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b81993d0 con 0x7fe7b8107270 2024-04-25T12:17:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.988+0000 7fe7b4ff9640 1 -- 172.21.15.19:0/2640526403 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0004d90 con 0x7fe7b8107270 2024-04-25T12:17:56.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.988+0000 7fe7bd272640 1 -- 172.21.15.19:0/2640526403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b81a55d0 con 0x7fe7b8107270 2024-04-25T12:17:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.989+0000 7fe7bd272640 1 -- 172.21.15.19:0/2640526403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b80691c0 con 0x7fe7b8107270 2024-04-25T12:17:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.990+0000 7fe7b4ff9640 1 -- 172.21.15.19:0/2640526403 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe7a0004f30 con 0x7fe7b8107270 2024-04-25T12:17:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.991+0000 7fe7b4ff9640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fe7a40a3d20 0x7fe7a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T12:17:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.991+0000 7fe7b6ffd640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fe7a40a3d20 0x7fe7a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T12:17:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.991+0000 7fe7b4ff9640 1 -- 172.21.15.19:0/2640526403 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8374+0+0 (secure 0 0 0) 0x7fe7a00d01b0 con 0x7fe7b8107270 2024-04-25T12:17:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.991+0000 7fe7b6ffd640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fe7a40a3d20 0x7fe7a40a61e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8006010 tx=0x7fe7a8000bc0 comp rx=0 tx=0).ready entity=mgr.15381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T12:17:56.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:56.996+0000 7fe7b4ff9640 1 -- 172.21.15.19:0/2640526403 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7a009c880 con 0x7fe7b8107270 2024-04-25T12:17:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:57.182+0000 7fe7bd272640 1 -- 172.21.15.19:0/2640526403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe7b8199af0 con 0x7fe7b8107270 2024-04-25T12:17:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:57.859+0000 7fe7b4ff9640 1 -- 172.21.15.19:0/2640526403 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) ==== 169+0+0 (secure 0 0 0) 0x7fe7b8199af0 con 0x7fe7b8107270 2024-04-25T12:17:57.861 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-04-25T12:17:57.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:57.864+0000 7fe7bd272640 1 -- 172.21.15.19:0/2640526403 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fe7a40a3d20 msgr2=0x7fe7a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:57.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:57.864+0000 7fe7bd272640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fe7a40a3d20 0x7fe7a40a61e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8006010 tx=0x7fe7a8000bc0 comp rx=0 tx=0).stop 2024-04-25T12:17:57.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:57.864+0000 7fe7bd272640 1 -- 172.21.15.19:0/2640526403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7b8107270 msgr2=0x7fe7b819e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T12:17:57.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:57.864+0000 7fe7bd272640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7b8107270 0x7fe7b819e950 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000d740 tx=0x7fe7a000dc00 comp rx=0 tx=0).stop 2024-04-25T12:17:57.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:57.864+0000 7fe7bd272640 1 -- 172.21.15.19:0/2640526403 shutdown_connections 2024-04-25T12:17:57.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:57.864+0000 7fe7bd272640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.19:6800/3657854393,v1:172.21.15.19:6801/3657854393] conn(0x7fe7a40a3d20 0x7fe7a40a61e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:57.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:57.864+0000 7fe7bd272640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b81985c0 0x7fe7b8198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:57.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:57.864+0000 7fe7bd272640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7b8107270 0x7fe7b819e950 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:57.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:57.864+0000 7fe7bd272640 1 --2- 172.21.15.19:0/2640526403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7b806a840 0x7fe7b819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T12:17:57.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:57.864+0000 7fe7bd272640 1 -- 172.21.15.19:0/2640526403 >> 172.21.15.19:0/2640526403 conn(0x7fe7b8077eb0 msgr2=0x7fe7b80fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T12:17:57.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:57.865+0000 7fe7bd272640 1 -- 172.21.15.19:0/2640526403 shutdown_connections 2024-04-25T12:17:57.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-04-25T12:17:57.865+0000 7fe7bd272640 1 -- 172.21.15.19:0/2640526403 wait complete. 2024-04-25T12:17:57.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T12:17:57.909 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T12:17:57.941 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T12:17:57.971 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T12:17:57.971 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T12:17:57.988 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T12:17:58.005 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T12:17:58.027 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T12:17:58.027 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T12:17:58.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mon.a 2024-04-25T12:17:58.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:57 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2479288829' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T12:17:58.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:57 smithi169 ceph-mon[32710]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T12:17:58.159 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:17:57 smithi169 ceph-mon[32710]: from='client.? 172.21.15.19:0/2640526403' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T12:17:58.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:57 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2479288829' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T12:17:58.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:57 smithi150 ceph-mon[32250]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T12:17:58.167 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:57 smithi150 ceph-mon[32250]: from='client.? 172.21.15.19:0/2640526403' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T12:17:58.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:57 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2479288829' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T12:17:58.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:57 smithi019 ceph-mon[29213]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T12:17:58.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:57 smithi019 ceph-mon[29213]: from='client.? 172.21.15.19:0/2640526403' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T12:17:58.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:58 smithi019 systemd[1]: Stopping Ceph mon.a for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:17:58.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:58 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a[29209]: 2024-04-25T12:17:58.326+0000 7fe73c57d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:17:58.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:58 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a[29209]: 2024-04-25T12:17:58.326+0000 7fe73c57d700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T12:17:58.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:58 smithi019 podman[87816]: 2024-04-25 12:17:58.348035609 +0000 UTC m=+0.123523181 container died 4a9fc10a4dcadd2afdfde1007aae6e54a191cc2eb2cfc031932e4fa418cce0fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T12:17:59.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:58 smithi019 podman[87816]: 2024-04-25 12:17:58.760633499 +0000 UTC m=+0.536121112 container cleanup 4a9fc10a4dcadd2afdfde1007aae6e54a191cc2eb2cfc031932e4fa418cce0fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T12:17:59.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:Apr 25 12:17:58 smithi019 bash[87816]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-a 2024-04-25T12:17:59.191 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T12:17:59.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:17:59.240 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T12:17:59.241 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T12:17:59.241 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mon.b 2024-04-25T12:17:59.561 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:59 smithi150 systemd[1]: Stopping Ceph mon.b for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:17:59.916 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:59 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-b[32246]: 2024-04-25T12:17:59.559+0000 7f5e0ce67700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:17:59.917 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:59 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-b[32246]: 2024-04-25T12:17:59.559+0000 7f5e0ce67700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T12:17:59.917 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:17:59 smithi150 podman[68228]: 2024-04-25 12:17:59.804926441 +0000 UTC m=+0.369181715 container died 2062640205a6e7e7552e46310591a02151c61676b0a592b8b663ef03d3d1051b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T12:18:00.417 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:18:00 smithi150 podman[68228]: 2024-04-25 12:18:00.209805971 +0000 UTC m=+0.774061237 container cleanup 2062640205a6e7e7552e46310591a02151c61676b0a592b8b663ef03d3d1051b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T12:18:00.417 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 25 12:18:00 smithi150 bash[68228]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-b 2024-04-25T12:18:00.645 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T12:18:00.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:18:00.709 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T12:18:00.709 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T12:18:00.709 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mon.c 2024-04-25T12:18:00.800 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:18:00 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:18:00.356+0000 7f24c6b87700 -1 client.15417: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-04-25T12:18:00.801 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:18:00 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:18:00.358+0000 7f24c6b87700 -1 client.15417: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-04-25T12:18:01.023 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:18:00 smithi169 systemd[1]: Stopping Ceph mon.c for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:18:01.409 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:18:01 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-c[32706]: 2024-04-25T12:18:01.021+0000 7f2026234700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:18:01.409 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:18:01 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-c[32706]: 2024-04-25T12:18:01.021+0000 7f2026234700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T12:18:01.410 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:18:01 smithi169 podman[69828]: 2024-04-25 12:18:01.246752438 +0000 UTC m=+0.338263273 container died e2e3dded6e416cf7ae1d646a70c1c618d3df20954e5f7e5202ce919f62a81848 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-25T12:18:01.858 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:18:01 smithi169 podman[69828]: 2024-04-25 12:18:01.631064401 +0000 UTC m=+0.722575283 container cleanup e2e3dded6e416cf7ae1d646a70c1c618d3df20954e5f7e5202ce919f62a81848 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T12:18:01.858 INFO:journalctl@ceph.mon.c.smithi169.stdout:Apr 25 12:18:01 smithi169 bash[69828]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mon-c 2024-04-25T12:18:02.005 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T12:18:02.046 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:18:02.046 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T12:18:02.046 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T12:18:02.046 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mgr.x 2024-04-25T12:18:02.367 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:18:02 smithi019 systemd[1]: Stopping Ceph mgr.x for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:18:02.675 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:18:02 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x[29446]: 2024-04-25T12:18:02.365+0000 7f2560bef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:18:02.675 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:18:02 smithi019 podman[87936]: 2024-04-25 12:18:02.39627081 +0000 UTC m=+0.151642603 container died 6340be20f2fab6dd4d54930113970627b028e74785bc889cd75e4eca0d823044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T12:18:03.051 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:18:02 smithi019 podman[87936]: 2024-04-25 12:18:02.782134433 +0000 UTC m=+0.537506271 container cleanup 6340be20f2fab6dd4d54930113970627b028e74785bc889cd75e4eca0d823044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T12:18:03.051 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Apr 25 12:18:02 smithi019 bash[87936]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-x 2024-04-25T12:18:03.200 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T12:18:03.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:18:03.236 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T12:18:03.237 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T12:18:03.237 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mgr.y 2024-04-25T12:18:03.602 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:18:03 smithi150 systemd[1]: Stopping Ceph mgr.y for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:18:03.908 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:18:03 smithi150 podman[68350]: 2024-04-25 12:18:03.602053896 +0000 UTC m=+0.171252619 container died 77cdcfc1595191b304d04177090607de609101c6de48440ca8ded64e4b179a27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T12:18:04.166 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:18:04 smithi150 podman[68350]: 2024-04-25 12:18:04.007100323 +0000 UTC m=+0.576299045 container cleanup 77cdcfc1595191b304d04177090607de609101c6de48440ca8ded64e4b179a27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T12:18:04.167 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 25 12:18:04 smithi150 bash[68350]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-y 2024-04-25T12:18:04.383 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T12:18:04.416 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:18:04.416 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T12:18:04.416 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T12:18:04.417 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mgr.z 2024-04-25T12:18:04.749 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:18:04 smithi169 systemd[1]: Stopping Ceph mgr.z for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:18:05.038 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:18:04 smithi169 podman[69950]: 2024-04-25 12:18:04.748673872 +0000 UTC m=+0.137739179 container died 631bf7fcd2a3d1c24ea2a6042b2a5f3a009681b198f6f0965af1227aef2bb073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T12:18:05.406 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:18:05 smithi169 podman[69950]: 2024-04-25 12:18:05.145524626 +0000 UTC m=+0.534589933 container cleanup 631bf7fcd2a3d1c24ea2a6042b2a5f3a009681b198f6f0965af1227aef2bb073 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T12:18:05.406 INFO:journalctl@ceph.mgr.z.smithi169.stdout:Apr 25 12:18:05 smithi169 bash[69950]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mgr-z 2024-04-25T12:18:05.562 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T12:18:05.595 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:18:05.596 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T12:18:05.596 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T12:18:05.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.0 2024-04-25T12:18:05.910 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:18:05 smithi019 systemd[1]: Stopping Ceph osd.0 for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:18:06.300 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:18:05 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-0[44023]: 2024-04-25T12:18:05.995+0000 7fa079d65700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:18:06.301 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:18:05 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-0[44023]: 2024-04-25T12:18:05.995+0000 7fa079d65700 -1 osd.0 99 *** Got signal Terminated *** 2024-04-25T12:18:06.301 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:18:05 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-0[44023]: 2024-04-25T12:18:05.995+0000 7fa079d65700 -1 osd.0 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T12:18:11.605 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:18:11 smithi019 podman[88059]: 2024-04-25 12:18:11.306460015 +0000 UTC m=+5.506019968 container died 647cfc1df8a85cf215d044c04ebbea564463ec7ce02d4ca4a5f144a397bb164c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T12:18:11.939 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:18:11 smithi019 podman[88059]: 2024-04-25 12:18:11.704404247 +0000 UTC m=+5.903964199 container cleanup 647cfc1df8a85cf215d044c04ebbea564463ec7ce02d4ca4a5f144a397bb164c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T12:18:11.939 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:18:11 smithi019 bash[88059]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-0 2024-04-25T12:18:12.240 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:18:11 smithi019 podman[88076]: 2024-04-25 12:18:11.938902063 +0000 UTC m=+0.623944892 container remove 647cfc1df8a85cf215d044c04ebbea564463ec7ce02d4ca4a5f144a397bb164c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True) 2024-04-25T12:18:12.510 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:18:12 smithi019 podman[88167]: 2024-04-25 12:18:12.142366748 +0000 UTC m=+0.018487699 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:18:12.510 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:18:12 smithi019 podman[88167]: 2024-04-25 12:18:12.264982812 +0000 UTC m=+0.141103805 container create ec63ccdac84c438e98fff8cde4c1a52411b5e0c31b45df82d28b64b0815f58b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T12:18:12.801 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:18:12 smithi019 podman[88167]: 2024-04-25 12:18:12.509308826 +0000 UTC m=+0.385429777 container init ec63ccdac84c438e98fff8cde4c1a52411b5e0c31b45df82d28b64b0815f58b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:18:12.801 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:18:12 smithi019 podman[88167]: 2024-04-25 12:18:12.512583608 +0000 UTC m=+0.388704561 container start ec63ccdac84c438e98fff8cde4c1a52411b5e0c31b45df82d28b64b0815f58b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T12:18:12.801 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:18:12 smithi019 podman[88167]: 2024-04-25 12:18:12.617919878 +0000 UTC m=+0.494040838 container attach ec63ccdac84c438e98fff8cde4c1a52411b5e0c31b45df82d28b64b0815f58b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T12:18:13.196 INFO:journalctl@ceph.osd.0.smithi019.stdout:Apr 25 12:18:12 smithi019 podman[88167]: 2024-04-25 12:18:12.831789685 +0000 UTC m=+0.707910638 container died ec63ccdac84c438e98fff8cde4c1a52411b5e0c31b45df82d28b64b0815f58b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-04-25T12:18:13.522 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T12:18:13.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:18:13.565 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T12:18:13.565 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T12:18:13.565 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.1 2024-04-25T12:18:13.878 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:18:13 smithi150 systemd[1]: Stopping Ceph osd.1 for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:18:14.166 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:18:13 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-1[36869]: 2024-04-25T12:18:13.902+0000 7fe2f9dff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:18:14.167 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:18:13 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-1[36869]: 2024-04-25T12:18:13.902+0000 7fe2f9dff700 -1 osd.1 99 *** Got signal Terminated *** 2024-04-25T12:18:14.167 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:18:13 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-1[36869]: 2024-04-25T12:18:13.902+0000 7fe2f9dff700 -1 osd.1 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T12:18:19.547 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:18:19 smithi150 podman[68476]: 2024-04-25 12:18:19.24176629 +0000 UTC m=+5.464569361 container died 580f7c13926ec37e5dd565694d205cae887df1ca29ad9a2b46162a3f3806317f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T12:18:19.916 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:18:19 smithi150 podman[68476]: 2024-04-25 12:18:19.64503034 +0000 UTC m=+5.867833411 container cleanup 580f7c13926ec37e5dd565694d205cae887df1ca29ad9a2b46162a3f3806317f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:18:19.917 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:18:19 smithi150 bash[68476]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-1 2024-04-25T12:18:20.264 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:18:19 smithi150 podman[68504]: 2024-04-25 12:18:19.940283014 +0000 UTC m=+0.693317501 container remove 580f7c13926ec37e5dd565694d205cae887df1ca29ad9a2b46162a3f3806317f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T12:18:20.543 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:18:20 smithi150 podman[68596]: 2024-04-25 12:18:20.165956624 +0000 UTC m=+0.017945681 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:18:20.543 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:18:20 smithi150 podman[68596]: 2024-04-25 12:18:20.297870875 +0000 UTC m=+0.149859915 container create 14a2f39626c5790576dbe9de9783ca9d89fd4dcf83b9357be013cef0c19804cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T12:18:20.543 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:18:20 smithi150 podman[68596]: 2024-04-25 12:18:20.542267871 +0000 UTC m=+0.394256911 container init 14a2f39626c5790576dbe9de9783ca9d89fd4dcf83b9357be013cef0c19804cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T12:18:20.839 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:18:20 smithi150 podman[68596]: 2024-04-25 12:18:20.545513331 +0000 UTC m=+0.397502374 container start 14a2f39626c5790576dbe9de9783ca9d89fd4dcf83b9357be013cef0c19804cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T12:18:20.839 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:18:20 smithi150 podman[68596]: 2024-04-25 12:18:20.617272151 +0000 UTC m=+0.469261206 container attach 14a2f39626c5790576dbe9de9783ca9d89fd4dcf83b9357be013cef0c19804cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:18:21.166 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 25 12:18:20 smithi150 podman[68596]: 2024-04-25 12:18:20.838821686 +0000 UTC m=+0.690810736 container died 14a2f39626c5790576dbe9de9783ca9d89fd4dcf83b9357be013cef0c19804cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:18:21.537 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T12:18:21.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:18:21.576 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T12:18:21.577 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T12:18:21.577 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.2 2024-04-25T12:18:21.894 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:18:21 smithi169 systemd[1]: Stopping Ceph osd.2 for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:18:22.159 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:18:21 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2[38026]: 2024-04-25T12:18:21.891+0000 7f1474b16700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:18:22.159 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:18:21 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2[38026]: 2024-04-25T12:18:21.891+0000 7f1474b16700 -1 osd.2 99 *** Got signal Terminated *** 2024-04-25T12:18:22.159 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:18:21 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2[38026]: 2024-04-25T12:18:21.891+0000 7f1474b16700 -1 osd.2 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T12:18:27.660 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:18:27 smithi169 podman[70089]: 2024-04-25 12:18:27.400862674 +0000 UTC m=+5.631696058 container died 2e96b3b5f20131d62adf939c8e3705cc03278a69f1b45a6f760708e5dee4e761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-04-25T12:18:28.159 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:18:27 smithi169 podman[70089]: 2024-04-25 12:18:27.750933708 +0000 UTC m=+5.981767078 container cleanup 2e96b3b5f20131d62adf939c8e3705cc03278a69f1b45a6f760708e5dee4e761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-25T12:18:28.159 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:18:27 smithi169 bash[70089]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2 2024-04-25T12:18:28.159 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:18:27 smithi169 podman[70107]: 2024-04-25 12:18:27.927524867 +0000 UTC m=+0.521152289 container remove 2e96b3b5f20131d62adf939c8e3705cc03278a69f1b45a6f760708e5dee4e761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T12:18:28.615 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:18:28 smithi169 podman[70201]: 2024-04-25 12:18:28.248885545 +0000 UTC m=+0.018359023 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:18:28.616 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:18:28 smithi169 podman[70201]: 2024-04-25 12:18:28.370242422 +0000 UTC m=+0.139715899 container create 305501d87a4e9f16c2608dcb238badef7690878ea2c3b35f92df7a4817bf0af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T12:18:28.909 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:18:28 smithi169 podman[70201]: 2024-04-25 12:18:28.614980949 +0000 UTC m=+0.384454427 container init 305501d87a4e9f16c2608dcb238badef7690878ea2c3b35f92df7a4817bf0af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T12:18:28.909 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:18:28 smithi169 podman[70201]: 2024-04-25 12:18:28.618137872 +0000 UTC m=+0.387611353 container start 305501d87a4e9f16c2608dcb238badef7690878ea2c3b35f92df7a4817bf0af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416) 2024-04-25T12:18:28.909 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:18:28 smithi169 podman[70201]: 2024-04-25 12:18:28.706652139 +0000 UTC m=+0.476125628 container attach 305501d87a4e9f16c2608dcb238badef7690878ea2c3b35f92df7a4817bf0af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T12:18:29.268 INFO:journalctl@ceph.osd.2.smithi169.stdout:Apr 25 12:18:28 smithi169 podman[70201]: 2024-04-25 12:18:28.910805182 +0000 UTC m=+0.680278661 container died 305501d87a4e9f16c2608dcb238badef7690878ea2c3b35f92df7a4817bf0af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T12:18:29.557 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T12:18:29.596 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:18:29.596 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T12:18:29.596 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T12:18:29.597 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.3 2024-04-25T12:18:29.917 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:18:29 smithi019 systemd[1]: Stopping Ceph osd.3 for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:18:30.198 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:18:29 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-3[50980]: 2024-04-25T12:18:29.915+0000 7f333321f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:18:30.199 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:18:29 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-3[50980]: 2024-04-25T12:18:29.915+0000 7f333321f700 -1 osd.3 99 *** Got signal Terminated *** 2024-04-25T12:18:30.199 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:18:29 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-3[50980]: 2024-04-25T12:18:29.915+0000 7f333321f700 -1 osd.3 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T12:18:31.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:31 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:31.487+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:32.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:32 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:32.466+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:32.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:32 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:32.530+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:33.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:33 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:33.097+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:33.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:33 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:33.513+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:33.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:33 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:33.486+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:33.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:33 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:33.529+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:34.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:34 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:34.069+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:34.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:34 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:34.472+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:34.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:34 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:34.531+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:34.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:34 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:34.529+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:35.104 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:34 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:34.719+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:35.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:35 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:35.103+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:35.535 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:35 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:35.506+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:35.535 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:35 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:35.471+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:35.536 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:18:35 smithi019 podman[88275]: 2024-04-25 12:18:35.263389601 +0000 UTC m=+5.469746124 container died afa45ec27c7d5cc2bbc7d50d539073e53e43803c0e333b4db26d2a9b13d24bff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-3, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T12:18:35.801 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:18:35 smithi019 podman[88275]: 2024-04-25 12:18:35.632902802 +0000 UTC m=+5.839259314 container cleanup afa45ec27c7d5cc2bbc7d50d539073e53e43803c0e333b4db26d2a9b13d24bff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:18:35.801 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:18:35 smithi019 bash[88275]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-3 2024-04-25T12:18:35.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:35 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:35.480+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:36.088 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:35 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:35.675+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:36.301 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:18:35 smithi019 podman[88302]: 2024-04-25 12:18:35.950989638 +0000 UTC m=+0.679686454 container remove afa45ec27c7d5cc2bbc7d50d539073e53e43803c0e333b4db26d2a9b13d24bff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:18:36.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:36 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:36.086+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:36.605 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:36 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:36.518+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:36.605 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:36 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:36.518+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:36.606 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:18:36 smithi019 podman[88392]: 2024-04-25 12:18:36.225924926 +0000 UTC m=+0.018744689 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:18:36.606 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:18:36 smithi019 podman[88392]: 2024-04-25 12:18:36.343638832 +0000 UTC m=+0.136458597 container create dd35e60c5f5afbe372cca4470c3782742d427a339e4b8807c7906189d68d7744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T12:18:36.606 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:18:36 smithi019 podman[88392]: 2024-04-25 12:18:36.604831286 +0000 UTC m=+0.397651050 container init dd35e60c5f5afbe372cca4470c3782742d427a339e4b8807c7906189d68d7744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-25T12:18:36.910 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:18:36 smithi019 podman[88392]: 2024-04-25 12:18:36.6081828 +0000 UTC m=+0.401002567 container start dd35e60c5f5afbe372cca4470c3782742d427a339e4b8807c7906189d68d7744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T12:18:36.910 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:18:36 smithi019 podman[88392]: 2024-04-25 12:18:36.713295647 +0000 UTC m=+0.506115428 container attach dd35e60c5f5afbe372cca4470c3782742d427a339e4b8807c7906189d68d7744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:18:36.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:36 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:36.453+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:37.075 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:36 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:36.713+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:37.226 INFO:journalctl@ceph.osd.3.smithi019.stdout:Apr 25 12:18:36 smithi019 podman[88392]: 2024-04-25 12:18:36.909960119 +0000 UTC m=+0.702779885 container died dd35e60c5f5afbe372cca4470c3782742d427a339e4b8807c7906189d68d7744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T12:18:37.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:37 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:37.074+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:37.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T12:18:37.615 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:18:37.615 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T12:18:37.615 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T12:18:37.615 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.4 2024-04-25T12:18:37.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:37 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:37.567+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:37.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:37 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:37.527+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:37.916 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:37 smithi150 systemd[1]: Stopping Ceph osd.4 for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:18:37.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:37 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:37.493+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:38.091 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:37 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:37.758+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:38.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:38 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:38.090+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:38.416 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:37 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4[43826]: 2024-04-25T12:18:37.932+0000 7fb3436e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:18:38.417 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:37 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4[43826]: 2024-04-25T12:18:37.932+0000 7fb3436e3700 -1 osd.4 99 *** Got signal Terminated *** 2024-04-25T12:18:38.417 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:37 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4[43826]: 2024-04-25T12:18:37.932+0000 7fb3436e3700 -1 osd.4 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T12:18:38.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:38 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:38.554+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:38.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:38 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:38.491+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:38.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:38 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:38.471+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:39.113 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:38 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:38.736+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:39.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:39 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:39.111+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:39.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:39 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:39.593+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:39.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:39 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:39.499+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:39.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:39 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:39.506+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:40.117 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:39 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:39.773+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:40.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:40 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:40.116+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:40.416 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:40 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4[43826]: 2024-04-25T12:18:40.156+0000 7fb33fcd7700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.972565+0000 front 2024-04-25T12:18:15.972519+0000 (oldest deadline 2024-04-25T12:18:40.072634+0000) 2024-04-25T12:18:40.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:40 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:40.467+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:40.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:40 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:40.467+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:41.051 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:40 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:40.550+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:41.051 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:40 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:40.558+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:41.051 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:40 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:40.558+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:41.151 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:40 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:40.724+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:41.151 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:40 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:40.724+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:41.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:41 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:41.149+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:41.476 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:41 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4[43826]: 2024-04-25T12:18:41.195+0000 7fb33fcd7700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.972565+0000 front 2024-04-25T12:18:15.972519+0000 (oldest deadline 2024-04-25T12:18:40.072634+0000) 2024-04-25T12:18:41.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:41 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:41.501+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:41.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:41 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:41.580+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:41.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:41 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:41.580+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:41.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:41 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:41.474+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:41.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:41 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:41.474+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:42.153 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:41 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:41.682+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:42.153 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:41 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:41.682+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:42.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:42 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:42.151+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:42.497 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:42 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4[43826]: 2024-04-25T12:18:42.218+0000 7fb33fcd7700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.972565+0000 front 2024-04-25T12:18:15.972519+0000 (oldest deadline 2024-04-25T12:18:40.072634+0000) 2024-04-25T12:18:42.772 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:42 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:42.495+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:42.772 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:42 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:42.495+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:42.800 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:42 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:42.532+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:42.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:42 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:42.532+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:42.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:42 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:42.628+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:42.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:42 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:42.628+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:42.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:42 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:42.639+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:42.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:42 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:42.639+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:43.490 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:43 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:43.207+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:43.491 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:43 smithi150 podman[68703]: 2024-04-25 12:18:43.430569342 +0000 UTC m=+5.617922088 container died 7dc2434adb5a3aeae18f6a3247f8447645ce54f0cd2f4e2f1542f37cbe9de14f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T12:18:43.659 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:43 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:43.175+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:43.751 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:43 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:43.488+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:43.751 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:43 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:43.488+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:43.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:43 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:43.649+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:43.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:43 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:43.649+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:43.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:43 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:43.538+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:43.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:43 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:43.538+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:44.150 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:43 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:43.668+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:44.151 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:43 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:43.668+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:44.151 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:43 smithi150 podman[68703]: 2024-04-25 12:18:43.866367915 +0000 UTC m=+6.053720668 container cleanup 7dc2434adb5a3aeae18f6a3247f8447645ce54f0cd2f4e2f1542f37cbe9de14f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T12:18:44.152 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:43 smithi150 bash[68703]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4 2024-04-25T12:18:44.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:44 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:44.150+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:44.416 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:44 smithi150 podman[68732]: 2024-04-25 12:18:44.1511824 +0000 UTC m=+0.722274110 container remove 7dc2434adb5a3aeae18f6a3247f8447645ce54f0cd2f4e2f1542f37cbe9de14f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-25T12:18:44.417 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:44 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:44.173+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:44.776 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:44 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:44.440+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:44.776 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:44 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:44.440+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:44.777 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:44 smithi150 podman[68827]: 2024-04-25 12:18:44.538688783 +0000 UTC m=+0.018139662 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:18:44.777 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:44 smithi150 podman[68827]: 2024-04-25 12:18:44.677219826 +0000 UTC m=+0.156670767 container create e0224da60b799ed156185b8b3072fac0055ddde508df83342e047085f8241e94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T12:18:45.051 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:44 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:44.673+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:45.051 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:44 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:44.673+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:45.051 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:44 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:44.564+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:45.051 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:44 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:44.564+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:45.060 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:44 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:44.699+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:45.060 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:44 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:44.699+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:45.139 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:44 smithi150 podman[68827]: 2024-04-25 12:18:44.904944098 +0000 UTC m=+0.384394978 container init e0224da60b799ed156185b8b3072fac0055ddde508df83342e047085f8241e94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T12:18:45.140 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:44 smithi150 podman[68827]: 2024-04-25 12:18:44.908108242 +0000 UTC m=+0.387559127 container start e0224da60b799ed156185b8b3072fac0055ddde508df83342e047085f8241e94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T12:18:45.140 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:45 smithi150 podman[68827]: 2024-04-25 12:18:45.004846897 +0000 UTC m=+0.484297787 container attach e0224da60b799ed156185b8b3072fac0055ddde508df83342e047085f8241e94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T12:18:45.400 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 25 12:18:45 smithi150 podman[68827]: 2024-04-25 12:18:45.204719244 +0000 UTC m=+0.684170115 container died e0224da60b799ed156185b8b3072fac0055ddde508df83342e047085f8241e94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T12:18:45.401 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:45 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:45.137+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:45.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:45 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:45.138+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:45.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:45 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:45.138+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.096988+0000 front 2024-04-25T12:18:18.097090+0000 (oldest deadline 2024-04-25T12:18:44.497627+0000) 2024-04-25T12:18:45.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:45 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:45.059+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:45.666 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:45 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:45.398+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:45.666 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:45 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:45.398+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:45.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:45 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:45.626+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:45.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:45 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:45.626+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:45.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:45 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:45.532+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:45.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:45 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:45.532+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:45.932 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T12:18:45.970 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:18:45.970 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T12:18:45.971 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T12:18:45.971 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.5 2024-04-25T12:18:45.998 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:45 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:45.740+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:45.998 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:45 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:45.740+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:46.273 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:46 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:46.082+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:46.273 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:46 smithi169 systemd[1]: Stopping Ceph osd.5 for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:18:46.274 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:46 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:46.181+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:46.274 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:46 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:46.181+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.096988+0000 front 2024-04-25T12:18:18.097090+0000 (oldest deadline 2024-04-25T12:18:44.497627+0000) 2024-04-25T12:18:46.274 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:46 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:46.271+0000 7f72d944e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:18:46.274 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:46 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:46.271+0000 7f72d944e700 -1 osd.5 99 *** Got signal Terminated *** 2024-04-25T12:18:46.659 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:46 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:46.271+0000 7f72d944e700 -1 osd.5 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T12:18:46.666 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:46 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:46.184+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:46.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:46 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:46.422+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:46.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:46 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:46.422+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:46.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:46 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:46.620+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:46.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:46 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:46.620+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:46.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:46 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:46.499+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:46.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:46 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:46.499+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:47.108 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:46 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:46.777+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:47.109 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:46 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:46.777+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:47.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:47 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:47.107+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:47.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:47 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:47.217+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:47.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:47 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:47.217+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.096988+0000 front 2024-04-25T12:18:18.097090+0000 (oldest deadline 2024-04-25T12:18:44.497627+0000) 2024-04-25T12:18:47.416 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:47 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:47.160+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:47.417 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:47 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:47.398+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:47.417 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:47 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:47.398+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:47.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:47 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:47.661+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:47.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:47 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:47.661+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:47.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:47 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:47.451+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:47.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:47 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:47.451+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:48.068 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:47 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:47.732+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:48.068 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:47 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:47.732+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:48.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:48 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:48.066+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:48.410 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:48 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:48.210+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:48.410 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:48 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:48.210+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.096988+0000 front 2024-04-25T12:18:18.097090+0000 (oldest deadline 2024-04-25T12:18:44.497627+0000) 2024-04-25T12:18:48.410 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:48 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:48.210+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.498304+0000 front 2024-04-25T12:18:24.498167+0000 (oldest deadline 2024-04-25T12:18:47.998290+0000) 2024-04-25T12:18:48.416 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:48 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:48.155+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:48.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:48 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:48.680+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:48.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:48 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:48.680+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:48.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:48 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:48.475+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:48.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:48 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:48.475+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:48.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:48 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:48.475+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:18:48.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:48 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:48.422+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:48.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:48 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:48.422+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:49.019 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:48 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:48.748+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:49.020 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:48 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:48.748+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:49.020 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:48 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:48.748+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:18:49.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:49 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:49.018+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:49.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:49 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:49.228+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:49.410 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:49 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:49.228+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.096988+0000 front 2024-04-25T12:18:18.097090+0000 (oldest deadline 2024-04-25T12:18:44.497627+0000) 2024-04-25T12:18:49.410 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:49 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:49.228+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.498304+0000 front 2024-04-25T12:18:24.498167+0000 (oldest deadline 2024-04-25T12:18:47.998290+0000) 2024-04-25T12:18:49.416 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:49 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:49.149+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:49.416 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:49 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:49.149+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:18:49.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:49 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:49.642+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:49.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:49 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:49.642+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:49.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:49 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:49.466+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:49.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:49 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:49.466+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:49.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:49 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:49.466+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:18:49.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:49 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:49.417+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:49.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:49 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:49.417+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:49.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:49 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:49.417+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:18:50.159 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:50 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:50.021+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:50.159 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:50 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:50.021+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:18:50.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:49 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:49.774+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:50.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:49 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:49.774+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:50.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:49 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:49.774+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:18:50.659 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:50 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:50.216+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:50.659 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:50 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:50.216+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.096988+0000 front 2024-04-25T12:18:18.097090+0000 (oldest deadline 2024-04-25T12:18:44.497627+0000) 2024-04-25T12:18:50.659 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:50 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:50.216+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.498304+0000 front 2024-04-25T12:18:24.498167+0000 (oldest deadline 2024-04-25T12:18:47.998290+0000) 2024-04-25T12:18:50.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:50 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:50.173+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:50.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:50 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:50.173+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:18:50.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:50 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:50.421+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:50.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:50 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:50.421+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:50.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:50 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:50.421+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:18:50.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:50 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:50.683+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:50.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:50 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:50.684+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:50.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:50 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:50.684+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:23.861478+0000 front 2024-04-25T12:18:23.861316+0000 (oldest deadline 2024-04-25T12:18:49.661810+0000) 2024-04-25T12:18:50.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:50 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:50.488+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:50.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:50 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:50.488+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:50.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:50 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:50.488+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:18:51.002 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:50 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:50.739+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:51.002 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:50 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:50.739+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:51.002 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:50 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:50.739+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:18:51.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:51 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:51.000+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:51.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:51 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:51.000+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:18:51.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:51 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:51.222+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.695720+0000 front 2024-04-25T12:18:08.695786+0000 (oldest deadline 2024-04-25T12:18:32.795907+0000) 2024-04-25T12:18:51.410 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:51 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:51.222+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.096988+0000 front 2024-04-25T12:18:18.097090+0000 (oldest deadline 2024-04-25T12:18:44.497627+0000) 2024-04-25T12:18:51.410 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:51 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5[44994]: 2024-04-25T12:18:51.222+0000 7f72d1a3a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.498304+0000 front 2024-04-25T12:18:24.498167+0000 (oldest deadline 2024-04-25T12:18:47.998290+0000) 2024-04-25T12:18:51.416 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:51 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:51.157+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:51.417 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:51 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:51.157+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:18:51.417 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:51 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:51.378+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:51.417 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:51 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:51.378+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:51.417 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:51 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:51.378+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:18:51.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:51 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:51.684+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:51.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:51 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:51.684+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:51.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:51 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:51.684+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:23.861478+0000 front 2024-04-25T12:18:23.861316+0000 (oldest deadline 2024-04-25T12:18:49.661810+0000) 2024-04-25T12:18:51.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:51 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:51.466+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:51.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:51 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:51.466+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:51.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:51 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:51.466+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:18:52.038 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:51 smithi169 podman[70332]: 2024-04-25 12:18:51.786339464 +0000 UTC m=+5.625618600 container died eea1c1d1e37dedf2469b53370d96539c3a97211030f30c8dbbe9cb8ef6e862dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T12:18:52.039 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:51 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:51.746+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:52.039 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:51 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:51.746+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:52.039 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:51 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:51.746+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:18:52.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:52 smithi169 podman[70332]: 2024-04-25 12:18:52.180578199 +0000 UTC m=+6.019857383 container cleanup eea1c1d1e37dedf2469b53370d96539c3a97211030f30c8dbbe9cb8ef6e862dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T12:18:52.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:52 smithi169 bash[70332]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5 2024-04-25T12:18:52.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:52 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:52.036+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:52.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:52 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:52.036+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:18:52.416 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:52 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:52.149+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:52.417 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:52 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:52.149+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:18:52.417 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:52 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:52.371+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:52.417 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:52 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:52.371+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:52.417 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:52 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:52.371+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:18:52.717 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:52 smithi169 podman[70350]: 2024-04-25 12:18:52.444865758 +0000 UTC m=+0.653524351 container remove eea1c1d1e37dedf2469b53370d96539c3a97211030f30c8dbbe9cb8ef6e862dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T12:18:52.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:52 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:52.648+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:52.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:52 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:52.648+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:52.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:52 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:52.648+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:23.861478+0000 front 2024-04-25T12:18:23.861316+0000 (oldest deadline 2024-04-25T12:18:49.661810+0000) 2024-04-25T12:18:52.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:52 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:52.512+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:52.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:52 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:52.512+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:52.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:52 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:52.512+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:18:53.037 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:52 smithi169 podman[70444]: 2024-04-25 12:18:52.700282553 +0000 UTC m=+0.018339209 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:18:53.037 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:52 smithi169 podman[70444]: 2024-04-25 12:18:52.916514856 +0000 UTC m=+0.234571497 container create caae8595c9a0a2dd5722dda8dd40c6337c09fa8f758cf09fc069db5a7dbed566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:18:53.037 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:52 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:52.715+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:53.037 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:52 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:52.715+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:53.037 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:52 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:52.715+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:18:53.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:53 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:53.061+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:53.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:53 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:53.061+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:18:53.409 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:53 smithi169 podman[70444]: 2024-04-25 12:18:53.179742132 +0000 UTC m=+0.497798770 container init caae8595c9a0a2dd5722dda8dd40c6337c09fa8f758cf09fc069db5a7dbed566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T12:18:53.410 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:53 smithi169 podman[70444]: 2024-04-25 12:18:53.182891198 +0000 UTC m=+0.500947836 container start caae8595c9a0a2dd5722dda8dd40c6337c09fa8f758cf09fc069db5a7dbed566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T12:18:53.410 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:53 smithi169 podman[70444]: 2024-04-25 12:18:53.27971078 +0000 UTC m=+0.597767431 container attach caae8595c9a0a2dd5722dda8dd40c6337c09fa8f758cf09fc069db5a7dbed566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T12:18:53.411 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:53 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:53.141+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:53.411 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:53 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:53.141+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:18:53.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:53 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:53.409+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:53.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:53 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:53.409+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:53.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:53 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:53.409+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:18:53.744 INFO:journalctl@ceph.osd.5.smithi169.stdout:Apr 25 12:18:53 smithi169 podman[70444]: 2024-04-25 12:18:53.489604056 +0000 UTC m=+0.807660688 container died caae8595c9a0a2dd5722dda8dd40c6337c09fa8f758cf09fc069db5a7dbed566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:18:53.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:53 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:53.631+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:53.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:53 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:53.631+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:53.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:53 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:53.631+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:23.861478+0000 front 2024-04-25T12:18:23.861316+0000 (oldest deadline 2024-04-25T12:18:49.661810+0000) 2024-04-25T12:18:53.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:53 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:53.466+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:53.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:53 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:53.466+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:53.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:53 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:53.466+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:18:54.027 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:53 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:53.742+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:54.027 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:53 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:53.742+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:54.027 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:53 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:53.742+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:18:54.081 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T12:18:54.160 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:18:54.160 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T12:18:54.160 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T12:18:54.160 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.6 2024-04-25T12:18:54.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:54.025+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:54.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:54.025+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:18:54.417 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:54.153+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:54.417 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:54.153+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:18:54.417 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:54.396+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:54.417 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:54.396+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:54.417 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:54 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:54.396+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:18:54.459 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:54 smithi019 systemd[1]: Stopping Ceph osd.6 for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:18:54.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:54.457+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:54.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:54.457+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:54.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:54.457+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:18:54.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:54.468+0000 7f1c6d2a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:18:54.802 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:54.468+0000 7f1c6d2a8700 -1 osd.6 99 *** Got signal Terminated *** 2024-04-25T12:18:54.802 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:54.468+0000 7f1c6d2a8700 -1 osd.6 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T12:18:54.802 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:54.607+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:54.802 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:54.607+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:54.802 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:54 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:54.607+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:23.861478+0000 front 2024-04-25T12:18:23.861316+0000 (oldest deadline 2024-04-25T12:18:49.661810+0000) 2024-04-25T12:18:55.057 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:54.702+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:55.058 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:54.702+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:55.058 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:54 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:54.702+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:18:55.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:55 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:55.055+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:55.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:55 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:55.055+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:18:55.416 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:55 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:55.164+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:55.416 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:55 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:55.164+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:18:55.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:55 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:55.613+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:55.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:55 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:55.613+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:55.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:55 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:55.613+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:23.861478+0000 front 2024-04-25T12:18:23.861316+0000 (oldest deadline 2024-04-25T12:18:49.661810+0000) 2024-04-25T12:18:55.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:55 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:55.445+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:55.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:55 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:55.445+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:55.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:55 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:55.445+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:18:55.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:55 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:55.445+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:55.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:55 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:55.445+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:55.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:55 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:55.445+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:18:56.013 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:55 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:55.667+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:56.014 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:55 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:55.667+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:56.014 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:55 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:55.667+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:18:56.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:56 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:56.012+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:56.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:56 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:56.012+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:18:56.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:56 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:56.012+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:18:56.666 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:56 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:56.206+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:56.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:56 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:56.206+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:18:56.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:56 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:56.400+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:56.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:56 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:56.400+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:56.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:56 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:56.400+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:18:56.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:56 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:56.400+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:18:56.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:56 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:56.585+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:56.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:56 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:56.585+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:56.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:56 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:56.585+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:23.861478+0000 front 2024-04-25T12:18:23.861316+0000 (oldest deadline 2024-04-25T12:18:49.661810+0000) 2024-04-25T12:18:56.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:56 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:56.421+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:56.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:56 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:56.421+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:56.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:56 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:56.421+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:18:56.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:56 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:56.421+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:18:56.966 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:56 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:56.688+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:56.966 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:56 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:56.688+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:56.967 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:56 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:56.688+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:18:57.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:56 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:56.965+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:57.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:56 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:56.965+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:18:57.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:56 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:56.965+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:18:57.416 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:57 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:57.163+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:57.416 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:57 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:57.163+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:18:57.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:57 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:57.595+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:57.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:57 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:57.595+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:57.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:57 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:57.595+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:23.861478+0000 front 2024-04-25T12:18:23.861316+0000 (oldest deadline 2024-04-25T12:18:49.661810+0000) 2024-04-25T12:18:57.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:57 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:57.595+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:31.962894+0000 front 2024-04-25T12:18:31.962931+0000 (oldest deadline 2024-04-25T12:18:56.662958+0000) 2024-04-25T12:18:57.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:57 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:57.407+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:57.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:57 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:57.407+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:57.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:57 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:57.407+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:18:57.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:57 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:57.407+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:18:57.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:57 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:57.424+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:57.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:57 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:57.424+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:57.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:57 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:57.424+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:18:57.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:57 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:57.424+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:18:58.002 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:57 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:57.726+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:58.002 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:57 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:57.726+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:58.002 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:57 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:57.726+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:18:58.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:58 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:58.000+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:58.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:58 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:58.000+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:18:58.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:58 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:58.000+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:18:58.416 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:58 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:58.160+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:58.416 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:58 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:58.160+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:18:58.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:58 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:58.557+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:08.158555+0000 front 2024-04-25T12:18:08.158607+0000 (oldest deadline 2024-04-25T12:18:33.458508+0000) 2024-04-25T12:18:58.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:58 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:58.557+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:15.759614+0000 front 2024-04-25T12:18:15.759691+0000 (oldest deadline 2024-04-25T12:18:39.859572+0000) 2024-04-25T12:18:58.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:58 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:58.557+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:23.861478+0000 front 2024-04-25T12:18:23.861316+0000 (oldest deadline 2024-04-25T12:18:49.661810+0000) 2024-04-25T12:18:58.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:58 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6[57868]: 2024-04-25T12:18:58.557+0000 7f1c6989c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:31.962894+0000 front 2024-04-25T12:18:31.962931+0000 (oldest deadline 2024-04-25T12:18:56.662958+0000) 2024-04-25T12:18:58.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:58 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:58.369+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:58.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:58 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:58.369+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:58.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:58 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:58.369+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:18:58.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:58 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:58.369+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:18:58.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:58 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:58.444+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:58.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:58 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:58.444+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:58.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:58 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:58.444+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:18:58.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:58 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:58.444+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:18:59.159 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:58 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:58.959+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:18:59.159 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:58 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:58.959+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:18:59.159 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:58 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:58.959+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:18:59.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:58 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:58.749+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:18:59.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:58 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:58.749+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:18:59.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:58 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:58.749+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:18:59.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:59 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:59.205+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:18:59.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:18:59 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:18:59.205+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:18:59.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:59 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:59.410+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:18:59.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:59 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:59.410+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:18:59.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:59 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:59.410+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:18:59.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:18:59 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:18:59.410+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:18:59.702 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:59 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:59.387+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:18:59.702 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:59 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:59.387+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:18:59.702 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:59 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:59.387+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:18:59.703 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:18:59 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:18:59.387+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:18:59.996 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:18:59 smithi019 podman[88500]: 2024-04-25 12:18:59.829334155 +0000 UTC m=+5.472842469 container died 24e9f2ad931d2f204c57dbc9f68593d77a8bad285a565a74a4740754450485f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T12:19:00.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:59 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:59.751+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:00.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:59 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:59.751+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:00.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:18:59 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:18:59.751+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:00.160 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:59 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:59.997+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:00.160 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:59 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:59.997+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:00.160 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:18:59 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:18:59.997+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:00.301 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:19:00 smithi019 podman[88500]: 2024-04-25 12:19:00.202518399 +0000 UTC m=+5.846026702 container cleanup 24e9f2ad931d2f204c57dbc9f68593d77a8bad285a565a74a4740754450485f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T12:19:00.301 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:19:00 smithi019 bash[88500]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6 2024-04-25T12:19:00.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:00 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:00.232+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:19:00.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:00 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:00.232+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:19:00.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:00 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:00.385+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:00.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:00 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:00.385+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:00.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:00 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:00.385+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:00.668 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:00 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:00.385+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:00.801 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:19:00 smithi019 podman[88518]: 2024-04-25 12:19:00.503997512 +0000 UTC m=+0.667102457 container remove 24e9f2ad931d2f204c57dbc9f68593d77a8bad285a565a74a4740754450485f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T12:19:00.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:00 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:00.395+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:00.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:00 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:00.395+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:00.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:00 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:00.395+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:00.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:00 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:00.395+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:01.108 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:19:00 smithi019 podman[88619]: 2024-04-25 12:19:00.715012793 +0000 UTC m=+0.018421373 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:19:01.108 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:19:00 smithi019 podman[88619]: 2024-04-25 12:19:00.846499121 +0000 UTC m=+0.149907690 container create 1f0c916b295234a2401fa48c511f0c046a1996073f3f7697dc8100cdf5dfb9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T12:19:01.159 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:01 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:01.022+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:01.159 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:01 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:01.022+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:01.159 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:01 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:01.022+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:01.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:00 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:00.793+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:01.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:00 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:00.793+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:01.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:00 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:00.793+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:01.417 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:19:01 smithi019 podman[88619]: 2024-04-25 12:19:01.107429857 +0000 UTC m=+0.410838428 container init 1f0c916b295234a2401fa48c511f0c046a1996073f3f7697dc8100cdf5dfb9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True) 2024-04-25T12:19:01.417 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:19:01 smithi019 podman[88619]: 2024-04-25 12:19:01.110998795 +0000 UTC m=+0.414407365 container start 1f0c916b295234a2401fa48c511f0c046a1996073f3f7697dc8100cdf5dfb9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T12:19:01.418 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:19:01 smithi019 podman[88619]: 2024-04-25 12:19:01.20727285 +0000 UTC m=+0.510681440 container attach 1f0c916b295234a2401fa48c511f0c046a1996073f3f7697dc8100cdf5dfb9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T12:19:01.666 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:01 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:01.215+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:19:01.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:01 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:01.215+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:19:01.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:01 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:01.215+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:29.957390+0000 front 2024-04-25T12:18:29.957475+0000 (oldest deadline 2024-04-25T12:19:01.158042+0000) 2024-04-25T12:19:01.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:01 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:01.368+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:01.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:01 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:01.368+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:01.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:01 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:01.368+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:01.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:01 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:01.368+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:01.728 INFO:journalctl@ceph.osd.6.smithi019.stdout:Apr 25 12:19:01 smithi019 podman[88619]: 2024-04-25 12:19:01.416534541 +0000 UTC m=+0.719943116 container died 1f0c916b295234a2401fa48c511f0c046a1996073f3f7697dc8100cdf5dfb9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T12:19:01.728 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:01 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:01.419+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:01.728 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:01 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:01.419+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:01.728 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:01 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:01.419+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:01.728 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:01 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:01.419+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:02.043 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T12:19:02.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:19:02.086 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T12:19:02.086 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T12:19:02.086 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.7 2024-04-25T12:19:02.159 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:02 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:02.000+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:02.160 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:02 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:02.000+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:02.160 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:02 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:02.000+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:02.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:01 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:01.798+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:02.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:01 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:01.798+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:02.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:01 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:01.798+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:02.405 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:02 smithi150 systemd[1]: Stopping Ceph osd.7 for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:19:02.405 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:02 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:02.225+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:19:02.405 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:02 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:02.225+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:19:02.405 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:02 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:02.225+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:29.957390+0000 front 2024-04-25T12:18:29.957475+0000 (oldest deadline 2024-04-25T12:19:01.158042+0000) 2024-04-25T12:19:02.405 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:02 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:02.403+0000 7fb4eaa36700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:19:02.405 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:02 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:02.403+0000 7fb4eaa36700 -1 osd.7 99 *** Got signal Terminated *** 2024-04-25T12:19:02.406 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:02 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:02.346+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:02.406 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:02 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:02.346+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:02.406 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:02 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:02.346+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:02.406 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:02 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:02.346+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:02.666 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:02 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:02.403+0000 7fb4eaa36700 -1 osd.7 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T12:19:02.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:02 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:02.436+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:02.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:02 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:02.436+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:02.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:02 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:02.436+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:02.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:02 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:02.436+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:03.159 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:02 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:02.951+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:03.159 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:02 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:02.951+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:03.159 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:02 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:02.951+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:03.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:02 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:02.811+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:03.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:02 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:02.811+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:03.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:02 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:02.811+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:03.666 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:03 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:03.197+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:19:03.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:03 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:03.197+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:19:03.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:03 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:03.197+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:29.957390+0000 front 2024-04-25T12:18:29.957475+0000 (oldest deadline 2024-04-25T12:19:01.158042+0000) 2024-04-25T12:19:03.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:03 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:03.384+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:03.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:03 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:03.384+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:03.668 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:03 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:03.384+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:03.668 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:03 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:03.384+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:03.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:03 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:03.405+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:03.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:03 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:03.405+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:03.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:03 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:03.405+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:03.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:03 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:03.405+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:04.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:03 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:03.795+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:04.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:03 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:03.795+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:04.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:03 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:03.795+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:04.160 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:03 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:03.999+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:04.160 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:03 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:03.999+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:04.160 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:03 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:03.999+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:04.160 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:03 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:03.999+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:40.522709+0000 front 2024-04-25T12:18:40.522839+0000 (oldest deadline 2024-04-25T12:19:03.422727+0000) 2024-04-25T12:19:04.666 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:04.235+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:19:04.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:04.235+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:19:04.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:04.235+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:29.957390+0000 front 2024-04-25T12:18:29.957475+0000 (oldest deadline 2024-04-25T12:19:01.158042+0000) 2024-04-25T12:19:04.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:04.384+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:04.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:04.384+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:04.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:04.384+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:04.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:04.384+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:04.668 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:04.384+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:04.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:04 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:04.403+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:04.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:04 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:04.403+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:04.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:04 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:04.403+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:04.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:04 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:04.403+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:05.036 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:04 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:04.760+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:05.036 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:04 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:04.760+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:05.036 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:04 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:04.760+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:05.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:05 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:05.034+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:05.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:05 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:05.034+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:05.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:05 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:05.034+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:05.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:05 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:05.034+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:40.522709+0000 front 2024-04-25T12:18:40.522839+0000 (oldest deadline 2024-04-25T12:19:03.422727+0000) 2024-04-25T12:19:05.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:05 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:05.225+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:19:05.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:05 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:05.225+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:19:05.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:05 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:05.225+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:29.957390+0000 front 2024-04-25T12:18:29.957475+0000 (oldest deadline 2024-04-25T12:19:01.158042+0000) 2024-04-25T12:19:05.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:05 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:05.408+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:05.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:05 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:05.408+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:05.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:05 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:05.408+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:05.668 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:05 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:05.408+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:05.668 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:05 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:05.408+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:05.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:05 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:05.410+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:05.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:05 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:05.410+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:05.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:05 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:05.410+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:05.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:05 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:05.410+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:05.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:05 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:05.410+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:06.056 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:05 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:05.711+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:06.056 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:05 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:05.711+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:06.056 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:05 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:05.711+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:06.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:06 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:06.054+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:06.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:06 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:06.054+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:06.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:06 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:06.054+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:06.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:06 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:06.054+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:40.522709+0000 front 2024-04-25T12:18:40.522839+0000 (oldest deadline 2024-04-25T12:19:03.422727+0000) 2024-04-25T12:19:06.666 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:06 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:06.180+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:19:06.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:06 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:06.180+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:19:06.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:06 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:06.180+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:29.957390+0000 front 2024-04-25T12:18:29.957475+0000 (oldest deadline 2024-04-25T12:19:01.158042+0000) 2024-04-25T12:19:06.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:06 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:06.414+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:06.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:06 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:06.414+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:06.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:06 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:06.414+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:06.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:06 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:06.414+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:06.668 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:06 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:06.414+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:06.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:06 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:06.363+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:06.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:06 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:06.363+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:06.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:06 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:06.363+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:06.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:06 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:06.363+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:06.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:06 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:06.363+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:07.082 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:06 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:06.722+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:07.083 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:06 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:06.722+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:07.083 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:06 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:06.722+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:07.083 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:06 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:06.722+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:07.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:07.081+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:07.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:07.081+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:07.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:07.081+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:07.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:07.081+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:40.522709+0000 front 2024-04-25T12:18:40.522839+0000 (oldest deadline 2024-04-25T12:19:03.422727+0000) 2024-04-25T12:19:07.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:07 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:07.219+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:17.655721+0000 front 2024-04-25T12:18:17.655805+0000 (oldest deadline 2024-04-25T12:18:42.355872+0000) 2024-04-25T12:19:07.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:07 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:07.219+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:22.356393+0000 front 2024-04-25T12:18:22.356329+0000 (oldest deadline 2024-04-25T12:18:48.256444+0000) 2024-04-25T12:19:07.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:07 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:07.219+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:29.957390+0000 front 2024-04-25T12:18:29.957475+0000 (oldest deadline 2024-04-25T12:19:01.158042+0000) 2024-04-25T12:19:07.667 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:07 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7[50834]: 2024-04-25T12:19:07.219+0000 7fb4e3022700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:41.158624+0000 front 2024-04-25T12:18:41.158551+0000 (oldest deadline 2024-04-25T12:19:07.058728+0000) 2024-04-25T12:19:07.668 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:07 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:07.450+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:07.668 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:07 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:07.450+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:07.668 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:07 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:07.450+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:07.668 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:07 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:07.450+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:07.668 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:07 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:07.450+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:07.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:07 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:07.343+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:07.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:07 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:07.343+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:07.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:07 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:07.343+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:07.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:07 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:07.343+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:07.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:07 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:07.343+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:08.087 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:07.745+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:08.088 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:07.745+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:08.088 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:07.745+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:08.088 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:07 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:07.745+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:08.166 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:07 smithi150 podman[68933]: 2024-04-25 12:19:07.886125601 +0000 UTC m=+5.590589150 container died 831b179dfb2287db959dd3b6ab30529144d349eee9e5927c1dd22100f25a6ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T12:19:08.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:08 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:08.086+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:08.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:08 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:08.086+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:08.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:08 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:08.086+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:08.410 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:08 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:08.086+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:40.522709+0000 front 2024-04-25T12:18:40.522839+0000 (oldest deadline 2024-04-25T12:19:03.422727+0000) 2024-04-25T12:19:08.538 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:08 smithi150 podman[68933]: 2024-04-25 12:19:08.311714074 +0000 UTC m=+6.016177620 container cleanup 831b179dfb2287db959dd3b6ab30529144d349eee9e5927c1dd22100f25a6ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T12:19:08.538 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:08 smithi150 bash[68933]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7 2024-04-25T12:19:08.539 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:08 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:08.416+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:08.539 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:08 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:08.416+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:08.539 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:08 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:08.416+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:08.539 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:08 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:08.416+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:08.539 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:08 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:08.416+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:08.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:08 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:08.298+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:08.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:08 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:08.298+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:08.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:08 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:08.298+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:08.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:08 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:08.298+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:08.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:08 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:08.298+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:08.917 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:08 smithi150 podman[68951]: 2024-04-25 12:19:08.537781126 +0000 UTC m=+0.654665822 container remove 831b179dfb2287db959dd3b6ab30529144d349eee9e5927c1dd22100f25a6ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:19:09.086 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:08 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:08.732+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:09.086 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:08 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:08.732+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:09.086 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:08 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:08.732+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:09.086 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:08 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:08.732+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:09.276 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:08 smithi150 podman[69044]: 2024-04-25 12:19:08.8883931 +0000 UTC m=+0.018115239 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:19:09.276 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:09 smithi150 podman[69044]: 2024-04-25 12:19:09.022287065 +0000 UTC m=+0.152009191 container create b9e2927844d7260a664078ce8fde515215be6ea4c37688849b8acc8d9e6d4a77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T12:19:09.276 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:09 smithi150 podman[69044]: 2024-04-25 12:19:09.275082548 +0000 UTC m=+0.404804675 container init b9e2927844d7260a664078ce8fde515215be6ea4c37688849b8acc8d9e6d4a77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T12:19:09.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:09.084+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:09.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:09.084+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:09.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:09.084+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:09.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:09.084+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:40.522709+0000 front 2024-04-25T12:18:40.522839+0000 (oldest deadline 2024-04-25T12:19:03.422727+0000) 2024-04-25T12:19:09.567 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:09 smithi150 podman[69044]: 2024-04-25 12:19:09.278267896 +0000 UTC m=+0.407990022 container start b9e2927844d7260a664078ce8fde515215be6ea4c37688849b8acc8d9e6d4a77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T12:19:09.567 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:09 smithi150 podman[69044]: 2024-04-25 12:19:09.366619089 +0000 UTC m=+0.496341225 container attach b9e2927844d7260a664078ce8fde515215be6ea4c37688849b8acc8d9e6d4a77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:19:09.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:09 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:09.413+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:09.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:09 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:09.413+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:09.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:09 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:09.413+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:09.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:09 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:09.413+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:09.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:09 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:09.413+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:09.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:09.347+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:09.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:09.347+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:09.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:09.347+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:09.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:09.347+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:09.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:09 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:09.347+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:09.870 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 25 12:19:09 smithi150 podman[69044]: 2024-04-25 12:19:09.567160313 +0000 UTC m=+0.696882440 container died b9e2927844d7260a664078ce8fde515215be6ea4c37688849b8acc8d9e6d4a77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T12:19:10.123 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:09.699+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:10.123 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:09.699+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:10.123 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:09.699+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:10.123 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:09 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:09.699+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:10.202 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T12:19:10.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:19:10.241 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T12:19:10.241 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T12:19:10.241 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.8 2024-04-25T12:19:10.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:10 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:10.121+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:10.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:10 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:10.121+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:10.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:10 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:10.121+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:10.410 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:10 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:10.121+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:40.522709+0000 front 2024-04-25T12:18:40.522839+0000 (oldest deadline 2024-04-25T12:19:03.422727+0000) 2024-04-25T12:19:10.410 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:10 smithi169 systemd[1]: Stopping Ceph osd.8 for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:19:10.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:10 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:10.355+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:10.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:10 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:10.355+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:10.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:10 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:10.355+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:10.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:10 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:10.355+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:10.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:10 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:10.355+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:10.909 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:10 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:10.543+0000 7f78a8f77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:19:10.909 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:10 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:10.543+0000 7f78a8f77700 -1 osd.8 99 *** Got signal Terminated *** 2024-04-25T12:19:10.909 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:10 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:10.543+0000 7f78a8f77700 -1 osd.8 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T12:19:10.910 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:10 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:10.693+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:10.910 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:10 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:10.693+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:10.910 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:10 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:10.693+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:10.910 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:10 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:10.693+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:10.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:10 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:10.462+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:10.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:10 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:10.462+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:10.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:10 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:10.462+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:10.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:10 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:10.462+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:10.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:10 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:10.462+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:11.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:11 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:11.141+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:11.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:11 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:11.141+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:11.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:11 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:11.141+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:11.409 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:11 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:11.141+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:40.522709+0000 front 2024-04-25T12:18:40.522839+0000 (oldest deadline 2024-04-25T12:19:03.422727+0000) 2024-04-25T12:19:11.666 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:11 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:11.477+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:11.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:11 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:11.477+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:11.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:11 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:11.477+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:11.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:11 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:11.477+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:11.667 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:11 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:11.477+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:11.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:11 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:11.331+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:11.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:11 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:11.331+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:11.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:11 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:11.331+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:11.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:11 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:11.331+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:11.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:11 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:11.331+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:11.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:11 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:11.648+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:11.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:11 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:11.648+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:11.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:11 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:11.648+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:11.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:11 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:11.648+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:12.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:12 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:12.282+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:12.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:12 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:12.282+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:12.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:12 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:12.282+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:12.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:12 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:12.282+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:12.552 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:12 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:12.282+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:12.601 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:12.173+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:12.602 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:12.173+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:12.602 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:12.173+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:12.602 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:12.173+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:40.522709+0000 front 2024-04-25T12:18:40.522839+0000 (oldest deadline 2024-04-25T12:19:03.422727+0000) 2024-04-25T12:19:12.874 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:12.600+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:12.874 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:12.600+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:12.874 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:12.600+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:12.874 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:12 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:12.600+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:12.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:12 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:12.516+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:12.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:12 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:12.516+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:12.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:12 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:12.516+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:12.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:12 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:12.516+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:12.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:12 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:12.516+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:13.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:13 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:13.243+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:13.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:13 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:13.243+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:13.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:13 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:13.243+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:13.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:13 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:13.243+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:13.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:13 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:13.243+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:13.643 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:13 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:13.216+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:13.643 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:13 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:13.216+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:13.643 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:13 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:13.216+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:13.643 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:13 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:13.216+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:40.522709+0000 front 2024-04-25T12:18:40.522839+0000 (oldest deadline 2024-04-25T12:19:03.422727+0000) 2024-04-25T12:19:13.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:13 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:13.641+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:13.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:13 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:13.641+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:13.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:13 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:13.641+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:13.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:13 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:13.641+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:13.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:13 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:13.641+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:13.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:13 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:13.554+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:13.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:13 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:13.554+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:13.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:13 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:13.554+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:13.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:13 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:13.554+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:13.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:13 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:13.554+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:13.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:13 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:13.554+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:14.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:14 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:14.201+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:14.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:14 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:14.201+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:14.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:14 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:14.201+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:14.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:14 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:14.201+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:14.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:14 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:14.201+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:14.618 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:14 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:14.194+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:14.618 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:14 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:14.194+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:14.619 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:14 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:14.194+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:14.619 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:14 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:14.194+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:40.522709+0000 front 2024-04-25T12:18:40.522839+0000 (oldest deadline 2024-04-25T12:19:03.422727+0000) 2024-04-25T12:19:14.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:14 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:14.616+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:14.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:14 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:14.616+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:14.910 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:14 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:14.616+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:14.910 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:14 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:14.616+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:14.910 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:14 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:14.616+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:14.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:14 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:14.519+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:14.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:14 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:14.519+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:14.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:14 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:14.519+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:14.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:14 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:14.519+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:14.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:14 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:14.519+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:14.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:14 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:14.519+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:15.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:15 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:15.209+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:15.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:15 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:15.209+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:15.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:15 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:15.209+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:15.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:15 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:15.209+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:15.552 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:15 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:15.209+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:15.659 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:15 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:15.236+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.319302+0000 front 2024-04-25T12:18:18.319248+0000 (oldest deadline 2024-04-25T12:18:44.219295+0000) 2024-04-25T12:19:15.659 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:15 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:15.236+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.219691+0000 front 2024-04-25T12:18:24.219718+0000 (oldest deadline 2024-04-25T12:18:49.519922+0000) 2024-04-25T12:19:15.659 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:15 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:15.236+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:30.620708+0000 front 2024-04-25T12:18:30.620903+0000 (oldest deadline 2024-04-25T12:18:55.920891+0000) 2024-04-25T12:19:15.660 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:15 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8[51829]: 2024-04-25T12:19:15.236+0000 7f78a1563700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:40.522709+0000 front 2024-04-25T12:18:40.522839+0000 (oldest deadline 2024-04-25T12:19:03.422727+0000) 2024-04-25T12:19:15.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:15 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:15.559+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:15.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:15 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:15.559+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:15.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:15 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:15.559+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:15.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:15 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:15.559+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:15.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:15 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:15.559+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:15.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:15 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:15.559+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:16.006 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:15 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:15.660+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:16.006 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:15 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:15.660+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:16.006 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:15 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:15.660+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:16.006 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:15 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:15.660+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:16.006 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:15 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:15.660+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:16.304 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:16 smithi169 podman[70552]: 2024-04-25 12:19:16.005328142 +0000 UTC m=+5.576239602 container died ed3ec86631bcea9deb9240fce3e68d5f1dd2ebb6b9280407f84db738b1f977ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T12:19:16.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:16 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:16.165+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:16.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:16 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:16.165+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:16.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:16 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:16.165+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:16.552 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:16 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:16.165+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:16.552 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:16 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:16.165+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:16.624 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:16 smithi169 podman[70552]: 2024-04-25 12:19:16.402193514 +0000 UTC m=+5.973104978 container cleanup ed3ec86631bcea9deb9240fce3e68d5f1dd2ebb6b9280407f84db738b1f977ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T12:19:16.624 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:16 smithi169 bash[70552]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8 2024-04-25T12:19:16.909 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:16 smithi169 podman[70581]: 2024-04-25 12:19:16.645159076 +0000 UTC m=+0.629242345 container remove ed3ec86631bcea9deb9240fce3e68d5f1dd2ebb6b9280407f84db738b1f977ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T12:19:16.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:16 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:16.622+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:16.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:16 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:16.622+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:16.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:16 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:16.622+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:16.910 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:16 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:16.622+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:16.910 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:16 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:16.622+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:16.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:16 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:16.580+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:16.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:16 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:16.580+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:16.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:16 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:16.580+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:16.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:16 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:16.580+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:16.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:16 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:16.580+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:16.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:16 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:16.580+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:17.258 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:16 smithi169 podman[70673]: 2024-04-25 12:19:16.872561613 +0000 UTC m=+0.018239084 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:19:17.258 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:16 smithi169 podman[70673]: 2024-04-25 12:19:16.995961545 +0000 UTC m=+0.141639046 container create 4e9b4f865254afd42cb64c92ef48abb3f4de99342c8c0d20476ca72c0eee8fea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:19:17.258 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:17 smithi169 podman[70673]: 2024-04-25 12:19:17.256835582 +0000 UTC m=+0.402513038 container init 4e9b4f865254afd42cb64c92ef48abb3f4de99342c8c0d20476ca72c0eee8fea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T12:19:17.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:17 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:17.209+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:17.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:17 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:17.209+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:17.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:17 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:17.209+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:17.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:17 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:17.209+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:17.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:17 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:17.209+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:17.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:17 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:17.209+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:49.774805+0000 front 2024-04-25T12:18:49.774609+0000 (oldest deadline 2024-04-25T12:19:16.174931+0000) 2024-04-25T12:19:17.555 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:17 smithi169 podman[70673]: 2024-04-25 12:19:17.259990574 +0000 UTC m=+0.405668031 container start 4e9b4f865254afd42cb64c92ef48abb3f4de99342c8c0d20476ca72c0eee8fea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T12:19:17.555 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:17 smithi169 podman[70673]: 2024-04-25 12:19:17.348359826 +0000 UTC m=+0.494037301 container attach 4e9b4f865254afd42cb64c92ef48abb3f4de99342c8c0d20476ca72c0eee8fea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T12:19:17.861 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:17 smithi169 podman[70673]: 2024-04-25 12:19:17.55445094 +0000 UTC m=+0.700128404 container died 4e9b4f865254afd42cb64c92ef48abb3f4de99342c8c0d20476ca72c0eee8fea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T12:19:17.861 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:17 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:17.637+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:17.862 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:17 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:17.637+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:17.862 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:17 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:17.637+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:17.862 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:17 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:17.637+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:17.862 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:17 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:17.637+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:17.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:17 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:17.625+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:17.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:17 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:17.625+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:17.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:17 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:17.625+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:17.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:17 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:17.625+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:17.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:17 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:17.625+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:17.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:17 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:17.625+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:18.113 INFO:journalctl@ceph.osd.8.smithi169.stdout:Apr 25 12:19:18 smithi169 podman[70698]: 2024-04-25 12:19:18.110541989 +0000 UTC m=+0.550398700 container remove 4e9b4f865254afd42cb64c92ef48abb3f4de99342c8c0d20476ca72c0eee8fea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T12:19:18.129 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T12:19:18.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:19:18.169 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T12:19:18.169 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T12:19:18.169 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.9 2024-04-25T12:19:18.483 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:18 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:18.212+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:18.483 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:18 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:18.212+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:18.483 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:18 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:18.212+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:18.484 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:18 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:18.212+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:18.484 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:18 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:18.212+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:18.484 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:18 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:18.212+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:49.774805+0000 front 2024-04-25T12:18:49.774609+0000 (oldest deadline 2024-04-25T12:19:16.174931+0000) 2024-04-25T12:19:18.484 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:18 smithi019 systemd[1]: Stopping Ceph osd.9 for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:19:18.484 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:18 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:18.481+0000 7f2d56484700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:19:18.484 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:18 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:18.481+0000 7f2d56484700 -1 osd.9 99 *** Got signal Terminated *** 2024-04-25T12:19:18.484 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:18 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:18.481+0000 7f2d56484700 -1 osd.9 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T12:19:18.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:18 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:18.614+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:18.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:18 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:18.614+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:18.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:18 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:18.614+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:18.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:18 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:18.614+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:18.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:18 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:18.614+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:18.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:18 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:18.614+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:19.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:18 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:18.678+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:19.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:18 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:18.678+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:19.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:18 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:18.678+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:19.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:18 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:18.678+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:19.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:18 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:18.678+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:19.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:19 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:19.243+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:19.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:19 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:19.243+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:19.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:19 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:19.243+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:19.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:19 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:19.243+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:19.552 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:19 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:19.243+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:19.552 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:19 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:19.243+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:49.774805+0000 front 2024-04-25T12:18:49.774609+0000 (oldest deadline 2024-04-25T12:19:16.174931+0000) 2024-04-25T12:19:19.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:19 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:19.645+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:19.909 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:19 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:19.645+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:19.910 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:19 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:19.645+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:19.910 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:19 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:19.645+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:19.910 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:19 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:19.645+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:19.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:19 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:19.592+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:19.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:19 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:19.592+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:19.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:19 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:19.592+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:19.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:19 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:19.592+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:19.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:19 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:19.592+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:19.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:19 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:19.592+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:20.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:20 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:20.272+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:20.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:20 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:20.272+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:20.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:20 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:20.272+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:20.551 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:20 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:20.272+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:20.552 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:20 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:20.272+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:20.552 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:20 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:20.272+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:49.774805+0000 front 2024-04-25T12:18:49.774609+0000 (oldest deadline 2024-04-25T12:19:16.174931+0000) 2024-04-25T12:19:20.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:20 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:20.642+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:20.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:20 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:20.642+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:20.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:20 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:20.642+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:20.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:20 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:20.642+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:20.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:20 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:20.642+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:20.918 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:20 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:20.642+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:21.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:20 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:20.688+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:21.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:20 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:20.688+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:21.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:20 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:20.688+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:21.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:20 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:20.688+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:21.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:20 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:20.688+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:21.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:21 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:21.317+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:21.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:21 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:21.317+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:21.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:21 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:21.317+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:21.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:21 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:21.317+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:21.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:21 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:21.317+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:21.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:21 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:21.317+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:49.774805+0000 front 2024-04-25T12:18:49.774609+0000 (oldest deadline 2024-04-25T12:19:16.174931+0000) 2024-04-25T12:19:21.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:21 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:21.661+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:21.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:21 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:21.661+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:21.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:21 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:21.661+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:21.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:21 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:21.661+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:21.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:21 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:21.661+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:21.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:21 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:21.661+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:22.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:21 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:21.662+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:22.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:21 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:21.662+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:22.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:21 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:21.662+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:22.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:21 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:21.662+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:22.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:21 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:21.662+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:22.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:21 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:21.662+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:22.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:22 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:22.328+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:22.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:22 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:22.328+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:22.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:22 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:22.328+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:22.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:22 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:22.328+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:22.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:22 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:22.328+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:22.803 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:22 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:22.328+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:49.774805+0000 front 2024-04-25T12:18:49.774609+0000 (oldest deadline 2024-04-25T12:19:16.174931+0000) 2024-04-25T12:19:22.916 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:22 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:22.633+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:22.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:22 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:22.633+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:22.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:22 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:22.633+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:22.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:22 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:22.633+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:22.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:22 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:22.633+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:22.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:22 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:22.633+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:22.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:22 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:22.633+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:58.745483+0000 front 2024-04-25T12:18:58.745540+0000 (oldest deadline 2024-04-25T12:19:22.245306+0000) 2024-04-25T12:19:23.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:22 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:22.694+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:23.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:22 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:22.694+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:23.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:22 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:22.694+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:23.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:22 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:22.694+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:23.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:22 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:22.694+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:23.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:22 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:22.694+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:23.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:23 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:23.333+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:06.765922+0000 front 2024-04-25T12:18:06.765979+0000 (oldest deadline 2024-04-25T12:18:31.466090+0000) 2024-04-25T12:19:23.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:23 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:23.333+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.467979+0000 front 2024-04-25T12:18:18.467775+0000 (oldest deadline 2024-04-25T12:18:41.967854+0000) 2024-04-25T12:19:23.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:23 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:23.333+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:21.968448+0000 front 2024-04-25T12:18:21.968502+0000 (oldest deadline 2024-04-25T12:18:47.868513+0000) 2024-04-25T12:19:23.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:23 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:23.333+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.372006+0000 front 2024-04-25T12:18:33.372064+0000 (oldest deadline 2024-04-25T12:18:55.672123+0000) 2024-04-25T12:19:23.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:23 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:23.333+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:39.173363+0000 front 2024-04-25T12:18:39.173287+0000 (oldest deadline 2024-04-25T12:19:04.473269+0000) 2024-04-25T12:19:23.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:23 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:23.333+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:49.774805+0000 front 2024-04-25T12:18:49.774609+0000 (oldest deadline 2024-04-25T12:19:16.174931+0000) 2024-04-25T12:19:23.802 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:23 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9[64913]: 2024-04-25T12:19:23.333+0000 7f2d4ea70700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.175334+0000 front 2024-04-25T12:18:56.175268+0000 (oldest deadline 2024-04-25T12:19:23.175934+0000) 2024-04-25T12:19:23.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:23 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:23.658+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:23.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:23 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:23.658+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:23.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:23 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:23.658+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:23.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:23 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:23.658+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:23.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:23 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:23.658+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:23.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:23 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:23.658+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:23.917 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:23 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:23.658+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:58.745483+0000 front 2024-04-25T12:18:58.745540+0000 (oldest deadline 2024-04-25T12:19:22.245306+0000) 2024-04-25T12:19:24.156 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:23 smithi019 podman[88725]: 2024-04-25 12:19:23.897644137 +0000 UTC m=+5.532978467 container died b6ee6182fab881c79baa2300f5e061915658892525f663ccf394ebfda1ec8646 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T12:19:24.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:23 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:23.736+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:24.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:23 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:23.736+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:24.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:23 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:23.736+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:24.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:23 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:23.736+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:24.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:23 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:23.736+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:24.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:23 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:23.736+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:24.439 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:24 smithi019 podman[88725]: 2024-04-25 12:19:24.245999028 +0000 UTC m=+5.881333366 container cleanup b6ee6182fab881c79baa2300f5e061915658892525f663ccf394ebfda1ec8646 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:19:24.440 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:24 smithi019 bash[88725]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9 2024-04-25T12:19:24.801 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:24 smithi019 podman[88743]: 2024-04-25 12:19:24.438737102 +0000 UTC m=+0.530213938 container remove b6ee6182fab881c79baa2300f5e061915658892525f663ccf394ebfda1ec8646 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:19:25.126 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:24 smithi019 podman[88835]: 2024-04-25 12:19:24.77095663 +0000 UTC m=+0.018316296 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:19:25.126 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:24 smithi019 podman[88835]: 2024-04-25 12:19:24.881275191 +0000 UTC m=+0.128634835 container create d225b98eee0cfa36412f5b320eb7b20cab7909f0c6f0f885352e51b119997957 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T12:19:25.126 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:25 smithi019 podman[88835]: 2024-04-25 12:19:25.125314288 +0000 UTC m=+0.372673926 container init d225b98eee0cfa36412f5b320eb7b20cab7909f0c6f0f885352e51b119997957 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T12:19:25.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:24 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:24.779+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:25.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:24 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:24.779+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:25.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:24 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:24.779+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:25.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:24 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:24.779+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:25.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:24 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:24.779+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:25.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:24 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:24.779+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:25.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:24 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:24.703+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:25.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:24 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:24.703+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:25.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:24 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:24.703+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:25.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:24 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:24.703+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:25.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:24 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:24.703+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:25.168 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:24 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:24.703+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:25.168 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:24 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:24.703+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:58.745483+0000 front 2024-04-25T12:18:58.745540+0000 (oldest deadline 2024-04-25T12:19:22.245306+0000) 2024-04-25T12:19:25.436 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:25 smithi019 podman[88835]: 2024-04-25 12:19:25.128695191 +0000 UTC m=+0.376054836 container start d225b98eee0cfa36412f5b320eb7b20cab7909f0c6f0f885352e51b119997957 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T12:19:25.436 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:25 smithi019 podman[88835]: 2024-04-25 12:19:25.208749943 +0000 UTC m=+0.456109586 container attach d225b98eee0cfa36412f5b320eb7b20cab7909f0c6f0f885352e51b119997957 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T12:19:25.770 INFO:journalctl@ceph.osd.9.smithi019.stdout:Apr 25 12:19:25 smithi019 podman[88835]: 2024-04-25 12:19:25.435384723 +0000 UTC m=+0.682744376 container died d225b98eee0cfa36412f5b320eb7b20cab7909f0c6f0f885352e51b119997957 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-9-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-25T12:19:26.063 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T12:19:26.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:19:26.105 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T12:19:26.105 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T12:19:26.105 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.10 2024-04-25T12:19:26.131 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:25 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:25.750+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:26.131 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:25 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:25.750+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:26.131 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:25 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:25.750+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:26.131 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:25 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:25.750+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:26.132 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:25 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:25.750+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:26.132 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:25 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:25.750+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:26.132 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:25 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:25.750+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:58.745483+0000 front 2024-04-25T12:18:58.745540+0000 (oldest deadline 2024-04-25T12:19:22.245306+0000) 2024-04-25T12:19:26.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:25 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:25.756+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:26.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:25 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:25.756+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:26.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:25 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:25.756+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:26.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:25 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:25.756+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:26.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:25 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:25.756+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:26.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:25 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:25.756+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:26.416 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:26 smithi150 systemd[1]: Stopping Ceph osd.10 for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:19:26.794 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:26 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:26.416+0000 7f7c4a9a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:19:26.794 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:26 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:26.416+0000 7f7c4a9a3700 -1 osd.10 99 *** Got signal Terminated *** 2024-04-25T12:19:26.794 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:26 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:26.416+0000 7f7c4a9a3700 -1 osd.10 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T12:19:27.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:26 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:26.734+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:27.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:26 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:26.734+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:27.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:26 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:26.734+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:27.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:26 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:26.734+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:27.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:26 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:26.734+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:27.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:26 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:26.734+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:27.166 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:26 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:26.793+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:27.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:26 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:26.793+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:27.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:26 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:26.793+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:27.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:26 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:26.793+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:27.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:26 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:26.793+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:27.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:26 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:26.793+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:27.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:26 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:26.793+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:58.745483+0000 front 2024-04-25T12:18:58.745540+0000 (oldest deadline 2024-04-25T12:19:22.245306+0000) 2024-04-25T12:19:28.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:27 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:27.755+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:28.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:27 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:27.755+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:28.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:27 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:27.755+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:28.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:27 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:27.755+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:28.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:27 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:27.755+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:28.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:27 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:27.755+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:28.166 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:27 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:27.817+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:28.166 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:27 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:27.817+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:28.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:27 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:27.817+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:28.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:27 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:27.817+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:28.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:27 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:27.817+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:28.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:27 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:27.817+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:28.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:27 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:27.817+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:58.745483+0000 front 2024-04-25T12:18:58.745540+0000 (oldest deadline 2024-04-25T12:19:22.245306+0000) 2024-04-25T12:19:29.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:28 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:28.728+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:29.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:28 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:28.728+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:29.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:28 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:28.728+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:29.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:28 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:28.728+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:29.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:28 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:28.728+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:29.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:28 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:28.728+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:29.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:28 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:28.799+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:29.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:28 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:28.799+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:29.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:28 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:28.799+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:29.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:28 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:28.799+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:29.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:28 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:28.799+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:29.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:28 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:28.799+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:29.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:28 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:28.799+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:58.745483+0000 front 2024-04-25T12:18:58.745540+0000 (oldest deadline 2024-04-25T12:19:22.245306+0000) 2024-04-25T12:19:30.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:29 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:29.683+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:30.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:29 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:29.683+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:30.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:29 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:29.683+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:30.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:29 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:29.683+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:30.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:29 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:29.683+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:30.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:29 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:29.683+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:30.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:29 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:29.683+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T12:19:04.322082+0000 front 2024-04-25T12:19:04.321950+0000 (oldest deadline 2024-04-25T12:19:29.621802+0000) 2024-04-25T12:19:30.166 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:29 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:29.809+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:30.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:29 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:29.809+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:30.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:29 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:29.809+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:30.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:29 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:29.809+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:30.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:29 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:29.809+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:30.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:29 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:29.809+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:30.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:29 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:29.809+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:58.745483+0000 front 2024-04-25T12:18:58.745540+0000 (oldest deadline 2024-04-25T12:19:22.245306+0000) 2024-04-25T12:19:31.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:30 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:30.718+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:31.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:30 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:30.718+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:31.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:30 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:30.718+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:31.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:30 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:30.718+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:31.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:30 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:30.718+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:31.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:30 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:30.718+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:31.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:30 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:30.718+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T12:19:04.322082+0000 front 2024-04-25T12:19:04.321950+0000 (oldest deadline 2024-04-25T12:19:29.621802+0000) 2024-04-25T12:19:31.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:30 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:30.796+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:09.937870+0000 front 2024-04-25T12:18:09.937838+0000 (oldest deadline 2024-04-25T12:18:32.237957+0000) 2024-04-25T12:19:31.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:30 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:30.796+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:18.639352+0000 front 2024-04-25T12:18:18.639329+0000 (oldest deadline 2024-04-25T12:18:39.739487+0000) 2024-04-25T12:19:31.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:30 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:30.796+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:25.640540+0000 front 2024-04-25T12:18:25.640376+0000 (oldest deadline 2024-04-25T12:18:49.140354+0000) 2024-04-25T12:19:31.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:30 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:30.796+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-04-25T12:18:33.741930+0000 front 2024-04-25T12:18:33.742026+0000 (oldest deadline 2024-04-25T12:18:56.041948+0000) 2024-04-25T12:19:31.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:30 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:30.797+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:38.843057+0000 front 2024-04-25T12:18:38.843108+0000 (oldest deadline 2024-04-25T12:19:03.543039+0000) 2024-04-25T12:19:31.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:30 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:30.797+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:47.644099+0000 front 2024-04-25T12:18:47.644132+0000 (oldest deadline 2024-04-25T12:19:13.543886+0000) 2024-04-25T12:19:31.167 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:30 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10[57615]: 2024-04-25T12:19:30.797+0000 7f7c42f8f700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:58.745483+0000 front 2024-04-25T12:18:58.745540+0000 (oldest deadline 2024-04-25T12:19:22.245306+0000) 2024-04-25T12:19:32.084 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:31 smithi150 podman[69161]: 2024-04-25 12:19:31.797802906 +0000 UTC m=+5.500119555 container died 17d5614afcf51be62c594bba691851007f90638da5a76e9a5c6a4504a142298d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T12:19:32.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:31 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:31.732+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:32.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:31 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:31.732+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:32.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:31 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:31.732+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:32.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:31 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:31.732+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:32.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:31 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:31.732+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:32.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:31 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:31.732+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:32.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:31 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:31.732+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T12:19:04.322082+0000 front 2024-04-25T12:19:04.321950+0000 (oldest deadline 2024-04-25T12:19:29.621802+0000) 2024-04-25T12:19:32.416 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:32 smithi150 podman[69161]: 2024-04-25 12:19:32.174257851 +0000 UTC m=+5.876574494 container cleanup 17d5614afcf51be62c594bba691851007f90638da5a76e9a5c6a4504a142298d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T12:19:32.417 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:32 smithi150 bash[69161]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10 2024-04-25T12:19:32.727 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:32 smithi150 podman[69178]: 2024-04-25 12:19:32.43364598 +0000 UTC m=+0.634581720 container remove 17d5614afcf51be62c594bba691851007f90638da5a76e9a5c6a4504a142298d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T12:19:33.055 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:32 smithi150 podman[69271]: 2024-04-25 12:19:32.630010914 +0000 UTC m=+0.018373406 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:19:33.055 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:32 smithi150 podman[69271]: 2024-04-25 12:19:32.742726095 +0000 UTC m=+0.131088579 container create 406efa8bc5d523f34d337d6c6450da504edb9fd709445614b4f8d0f4c5cb2bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T12:19:33.055 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:32 smithi150 podman[69271]: 2024-04-25 12:19:32.970521903 +0000 UTC m=+0.358884394 container init 406efa8bc5d523f34d337d6c6450da504edb9fd709445614b4f8d0f4c5cb2bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T12:19:33.055 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:32 smithi150 podman[69271]: 2024-04-25 12:19:32.973631558 +0000 UTC m=+0.361994039 container start 406efa8bc5d523f34d337d6c6450da504edb9fd709445614b4f8d0f4c5cb2bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T12:19:33.055 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:33 smithi150 podman[69271]: 2024-04-25 12:19:33.053855646 +0000 UTC m=+0.442218130 container attach 406efa8bc5d523f34d337d6c6450da504edb9fd709445614b4f8d0f4c5cb2bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-04-25T12:19:33.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:32 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:32.757+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:33.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:32 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:32.757+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:33.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:32 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:32.757+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:33.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:32 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:32.757+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:33.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:32 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:32.757+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:33.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:32 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:32.757+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:33.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:32 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:32.757+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T12:19:04.322082+0000 front 2024-04-25T12:19:04.321950+0000 (oldest deadline 2024-04-25T12:19:29.621802+0000) 2024-04-25T12:19:33.417 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 25 12:19:33 smithi150 podman[69271]: 2024-04-25 12:19:33.269565452 +0000 UTC m=+0.657927938 container died 406efa8bc5d523f34d337d6c6450da504edb9fd709445614b4f8d0f4c5cb2bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T12:19:33.887 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T12:19:33.964 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:19:33.964 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T12:19:33.965 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T12:19:33.965 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.11 2024-04-25T12:19:33.991 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:33 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:33.735+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:33.991 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:33 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:33.735+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:33.991 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:33 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:33.735+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:33.991 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:33 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:33.735+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:33.991 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:33 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:33.735+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:33.992 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:33 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:33.735+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:33.992 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:33 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:33.735+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T12:19:04.322082+0000 front 2024-04-25T12:19:04.321950+0000 (oldest deadline 2024-04-25T12:19:29.621802+0000) 2024-04-25T12:19:34.247 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:34 smithi169 systemd[1]: Stopping Ceph osd.11 for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:19:34.659 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:34 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:34.245+0000 7fab11924700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:19:34.659 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:34 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:34.245+0000 7fab11924700 -1 osd.11 99 *** Got signal Terminated *** 2024-04-25T12:19:34.659 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:34 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:34.245+0000 7fab11924700 -1 osd.11 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T12:19:35.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:34 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:34.761+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:35.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:34 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:34.761+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:35.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:34 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:34.761+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:35.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:34 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:34.761+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:35.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:34 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:34.761+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:35.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:34 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:34.761+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:35.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:34 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:34.761+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T12:19:04.322082+0000 front 2024-04-25T12:19:04.321950+0000 (oldest deadline 2024-04-25T12:19:29.621802+0000) 2024-04-25T12:19:36.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:35 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:35.736+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:36.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:35 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:35.736+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:36.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:35 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:35.736+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:36.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:35 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:35.736+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:36.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:35 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:35.736+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:36.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:35 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:35.736+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:36.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:35 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:35.736+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T12:19:04.322082+0000 front 2024-04-25T12:19:04.321950+0000 (oldest deadline 2024-04-25T12:19:29.621802+0000) 2024-04-25T12:19:37.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:36 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:36.779+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:37.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:36 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:36.779+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:37.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:36 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:36.779+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:37.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:36 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:36.779+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:37.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:36 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:36.779+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:37.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:36 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:36.779+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:37.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:36 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:36.779+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T12:19:04.322082+0000 front 2024-04-25T12:19:04.321950+0000 (oldest deadline 2024-04-25T12:19:29.621802+0000) 2024-04-25T12:19:37.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:36 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:36.779+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6820 osd.8 since back 2024-04-25T12:19:15.522623+0000 front 2024-04-25T12:19:15.522581+0000 (oldest deadline 2024-04-25T12:19:36.022763+0000) 2024-04-25T12:19:38.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:37 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:37.786+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:38.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:37 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:37.786+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:38.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:37 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:37.786+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:38.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:37 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:37.786+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:38.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:37 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:37.786+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:38.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:37 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:37.786+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:38.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:37 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:37.786+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T12:19:04.322082+0000 front 2024-04-25T12:19:04.321950+0000 (oldest deadline 2024-04-25T12:19:29.621802+0000) 2024-04-25T12:19:38.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:37 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:37.786+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6820 osd.8 since back 2024-04-25T12:19:15.522623+0000 front 2024-04-25T12:19:15.522581+0000 (oldest deadline 2024-04-25T12:19:36.022763+0000) 2024-04-25T12:19:39.159 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:38 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:38.773+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-04-25T12:18:10.412175+0000 front 2024-04-25T12:18:10.411987+0000 (oldest deadline 2024-04-25T12:18:34.512024+0000) 2024-04-25T12:19:39.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:38 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:38.773+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-25T12:18:14.512494+0000 front 2024-04-25T12:18:14.512416+0000 (oldest deadline 2024-04-25T12:18:39.812447+0000) 2024-04-25T12:19:39.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:38 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:38.773+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6804 osd.2 since back 2024-04-25T12:18:24.413855+0000 front 2024-04-25T12:18:24.414137+0000 (oldest deadline 2024-04-25T12:18:47.914126+0000) 2024-04-25T12:19:39.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:38 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:38.773+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-25T12:18:42.317910+0000 front 2024-04-25T12:18:42.318059+0000 (oldest deadline 2024-04-25T12:19:05.817695+0000) 2024-04-25T12:19:39.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:38 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:38.773+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6812 osd.5 since back 2024-04-25T12:18:51.019146+0000 front 2024-04-25T12:18:51.019173+0000 (oldest deadline 2024-04-25T12:19:13.219868+0000) 2024-04-25T12:19:39.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:38 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:38.773+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-04-25T12:18:56.120962+0000 front 2024-04-25T12:18:56.120978+0000 (oldest deadline 2024-04-25T12:19:21.420807+0000) 2024-04-25T12:19:39.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:38 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:38.773+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-25T12:19:04.322082+0000 front 2024-04-25T12:19:04.321950+0000 (oldest deadline 2024-04-25T12:19:29.621802+0000) 2024-04-25T12:19:39.160 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:38 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11[58855]: 2024-04-25T12:19:38.773+0000 7fab0df18700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.169:6820 osd.8 since back 2024-04-25T12:19:15.522623+0000 front 2024-04-25T12:19:15.522581+0000 (oldest deadline 2024-04-25T12:19:36.022763+0000) 2024-04-25T12:19:40.006 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:39 smithi169 podman[70779]: 2024-04-25 12:19:39.754513884 +0000 UTC m=+5.600798727 container died 8d4c3df65cae3aae6a378d13c89de36f94be5a69f6aa3ae8c1a233b285861ca5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T12:19:40.280 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:40 smithi169 podman[70779]: 2024-04-25 12:19:40.095176947 +0000 UTC m=+5.941461789 container cleanup 8d4c3df65cae3aae6a378d13c89de36f94be5a69f6aa3ae8c1a233b285861ca5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T12:19:40.281 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:40 smithi169 bash[70779]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11 2024-04-25T12:19:40.659 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:40 smithi169 podman[70795]: 2024-04-25 12:19:40.279632541 +0000 UTC m=+0.525629758 container remove 8d4c3df65cae3aae6a378d13c89de36f94be5a69f6aa3ae8c1a233b285861ca5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T12:19:41.017 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:40 smithi169 podman[70885]: 2024-04-25 12:19:40.618373183 +0000 UTC m=+0.017980762 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T12:19:41.018 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:40 smithi169 podman[70885]: 2024-04-25 12:19:40.73080144 +0000 UTC m=+0.130409010 container create 56326273b8df395a57c8d223c0d9160e37720d106be72225fb85659f42deca4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:19:41.018 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:40 smithi169 podman[70885]: 2024-04-25 12:19:40.95049308 +0000 UTC m=+0.350100659 container init 56326273b8df395a57c8d223c0d9160e37720d106be72225fb85659f42deca4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T12:19:41.018 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:40 smithi169 podman[70885]: 2024-04-25 12:19:40.953680628 +0000 UTC m=+0.353288208 container start 56326273b8df395a57c8d223c0d9160e37720d106be72225fb85659f42deca4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-25T12:19:41.018 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:41 smithi169 podman[70885]: 2024-04-25 12:19:41.016342695 +0000 UTC m=+0.415950270 container attach 56326273b8df395a57c8d223c0d9160e37720d106be72225fb85659f42deca4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T12:19:41.409 INFO:journalctl@ceph.osd.11.smithi169.stdout:Apr 25 12:19:41 smithi169 podman[70885]: 2024-04-25 12:19:41.255277862 +0000 UTC m=+0.654885432 container died 56326273b8df395a57c8d223c0d9160e37720d106be72225fb85659f42deca4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T12:19:41.793 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T12:19:41.869 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:19:41.869 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T12:19:41.870 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T12:19:41.870 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.a 2024-04-25T12:19:42.157 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:19:41 smithi019 systemd[1]: Stopping Ceph mds.a for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:19:42.553 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:19:42 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-a[71737]: 2024-04-25T12:19:42.155+0000 7f4c66628700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:19:42.554 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:19:42 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-a[71737]: 2024-04-25T12:19:42.155+0000 7f4c66628700 -1 mds.a *** got signal Terminated *** 2024-04-25T12:19:46.432 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:19:46 smithi019 podman[88951]: 2024-04-25 12:19:46.162709125 +0000 UTC m=+4.098744968 container died 765dadd9a116f93efc4f7545e22f589a4b48d600dc137de2d0ea650479dab066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T12:19:46.801 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:19:46 smithi019 podman[88951]: 2024-04-25 12:19:46.629975184 +0000 UTC m=+4.566011030 container cleanup 765dadd9a116f93efc4f7545e22f589a4b48d600dc137de2d0ea650479dab066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T12:19:46.801 INFO:journalctl@ceph.mds.a.smithi019.stdout:Apr 25 12:19:46 smithi019 bash[88951]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-a 2024-04-25T12:19:47.076 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T12:19:47.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:19:47.112 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T12:19:47.112 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T12:19:47.112 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.d 2024-04-25T12:19:47.452 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:19:47 smithi019 systemd[1]: Stopping Ceph mds.d for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:19:47.801 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:19:47 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-d[70773]: 2024-04-25T12:19:47.450+0000 7f3ae0ebd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:19:47.801 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:19:47 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-d[70773]: 2024-04-25T12:19:47.450+0000 7f3ae0ebd700 -1 mds.d *** got signal Terminated *** 2024-04-25T12:19:51.002 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:19:50 smithi019 podman[89072]: 2024-04-25 12:19:50.613867837 +0000 UTC m=+3.271713935 container died 3d49cfc6549a0f82515144f7ecabf973d284ba0166fc9e8665809faa5a61f5f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:19:51.301 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:19:51 smithi019 podman[89072]: 2024-04-25 12:19:51.099815859 +0000 UTC m=+3.757661956 container cleanup 3d49cfc6549a0f82515144f7ecabf973d284ba0166fc9e8665809faa5a61f5f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T12:19:51.301 INFO:journalctl@ceph.mds.d.smithi019.stdout:Apr 25 12:19:51 smithi019 bash[89072]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-d 2024-04-25T12:19:51.527 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T12:19:51.560 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:19:51.560 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T12:19:51.560 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T12:19:51.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.g 2024-04-25T12:19:51.801 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:19:51 smithi019 systemd[1]: Stopping Ceph mds.g for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:19:52.301 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:19:51 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-g[71385]: 2024-04-25T12:19:51.887+0000 7f607d347700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:19:52.301 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:19:51 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-g[71385]: 2024-04-25T12:19:51.887+0000 7f607d347700 -1 mds.g *** got signal Terminated *** 2024-04-25T12:19:55.301 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:19:54 smithi019 podman[89193]: 2024-04-25 12:19:54.908002249 +0000 UTC m=+3.124704062 container died ad1bcb2992053629f5bd891450e3be4505182c0d2c2b35e70e4d48dce7f9de1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T12:19:55.657 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:19:55 smithi019 podman[89193]: 2024-04-25 12:19:55.413597141 +0000 UTC m=+3.630298955 container cleanup ad1bcb2992053629f5bd891450e3be4505182c0d2c2b35e70e4d48dce7f9de1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T12:19:55.657 INFO:journalctl@ceph.mds.g.smithi019.stdout:Apr 25 12:19:55 smithi019 bash[89193]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-g 2024-04-25T12:19:55.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T12:19:55.832 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:19:55.833 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T12:19:55.833 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T12:19:55.833 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.j 2024-04-25T12:19:56.168 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:19:55 smithi019 systemd[1]: Stopping Ceph mds.j for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:19:56.551 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:19:56 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-j[70389]: 2024-04-25T12:19:56.166+0000 7f26fbadc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:19:56.551 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:19:56 smithi019 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-j[70389]: 2024-04-25T12:19:56.166+0000 7f26fbadc700 -1 mds.j *** got signal Terminated *** 2024-04-25T12:19:59.861 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:19:59 smithi019 podman[89316]: 2024-04-25 12:19:59.58780877 +0000 UTC m=+3.532314552 container died 66947b32c094f22f69abc2e938048312a0eebe7ca9cc2bf4934348512722fdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:20:00.244 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:19:59 smithi019 podman[89316]: 2024-04-25 12:19:59.942658562 +0000 UTC m=+3.887164343 container cleanup 66947b32c094f22f69abc2e938048312a0eebe7ca9cc2bf4934348512722fdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T12:20:00.244 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:19:59 smithi019 bash[89316]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-j 2024-04-25T12:20:00.483 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T12:20:00.568 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:20:00 smithi019 podman[89330]: 2024-04-25 12:20:00.243465188 +0000 UTC m=+0.644840065 container remove 66947b32c094f22f69abc2e938048312a0eebe7ca9cc2bf4934348512722fdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-04-25T12:20:00.569 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:20:00 smithi019 systemd[1]: ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.j.service: Deactivated successfully. 2024-04-25T12:20:00.569 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:20:00 smithi019 systemd[1]: Stopped Ceph mds.j for be62e290-02fa-11ef-bc93-c7b262605968. 2024-04-25T12:20:00.569 INFO:journalctl@ceph.mds.j.smithi019.stdout:Apr 25 12:20:00 smithi019 systemd[1]: ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.j.service: Consumed 1.123s CPU time. 2024-04-25T12:20:00.602 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:20:00.602 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T12:20:00.602 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T12:20:00.602 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.b 2024-04-25T12:20:01.176 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 12:20:00 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-b[62826]: 2024-04-25T12:20:00.982+0000 7fe1826fa700 -1 mds.b *** got signal Terminated *** 2024-04-25T12:20:03.167 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 12:20:02 smithi150 podman[69378]: 2024-04-25 12:20:02.909696006 +0000 UTC m=+2.118726046 container died 764b69f2b93849e247c40c8e339a691c57908082051d8129d51b246b1c74798e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T12:20:03.860 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 12:20:03 smithi150 podman[69378]: 2024-04-25 12:20:03.566922967 +0000 UTC m=+2.775953014 container cleanup 764b69f2b93849e247c40c8e339a691c57908082051d8129d51b246b1c74798e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T12:20:03.861 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 25 12:20:03 smithi150 bash[69378]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-b 2024-04-25T12:20:03.993 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T12:20:04.029 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:20:04.030 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T12:20:04.030 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T12:20:04.030 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.e 2024-04-25T12:20:04.167 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:20:04 smithi150 systemd[1]: Stopping Ceph mds.e for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:20:04.667 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:20:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-e[62211]: 2024-04-25T12:20:04.370+0000 7fd36db60700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:20:04.667 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:20:04 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-e[62211]: 2024-04-25T12:20:04.370+0000 7fd36db60700 -1 mds.e *** got signal Terminated *** 2024-04-25T12:20:07.167 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:20:06 smithi150 podman[69511]: 2024-04-25 12:20:06.912079429 +0000 UTC m=+2.663409116 container died 58a4be0bae3252b3ea38cc15c60102ee78652367a3f0444bf815b6947ccd2225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:20:07.651 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:20:07 smithi150 podman[69511]: 2024-04-25 12:20:07.238191055 +0000 UTC m=+2.989520736 container cleanup 58a4be0bae3252b3ea38cc15c60102ee78652367a3f0444bf815b6947ccd2225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T12:20:07.652 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:20:07 smithi150 bash[69511]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-e 2024-04-25T12:20:07.652 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 25 12:20:07 smithi150 podman[69526]: 2024-04-25 12:20:07.43941535 +0000 UTC m=+0.526859833 container remove 58a4be0bae3252b3ea38cc15c60102ee78652367a3f0444bf815b6947ccd2225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T12:20:07.669 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T12:20:07.700 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:20:07.701 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T12:20:07.701 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T12:20:07.701 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.h 2024-04-25T12:20:07.916 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:20:07 smithi150 systemd[1]: Stopping Ceph mds.h for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:20:08.417 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:20:08 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-h[62521]: 2024-04-25T12:20:08.075+0000 7fbb6997b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:20:08.417 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:20:08 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-h[62521]: 2024-04-25T12:20:08.075+0000 7fbb6997b700 -1 mds.h *** got signal Terminated *** 2024-04-25T12:20:11.167 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:20:10 smithi150 podman[69633]: 2024-04-25 12:20:10.915295395 +0000 UTC m=+2.998157844 container died 3c0afcfe116f5b36d7d7217a6d4d84d93792c1ec853c102d0fd589b03cb39403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T12:20:11.552 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:20:11 smithi150 podman[69633]: 2024-04-25 12:20:11.284247914 +0000 UTC m=+3.367110363 container cleanup 3c0afcfe116f5b36d7d7217a6d4d84d93792c1ec853c102d0fd589b03cb39403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-h, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T12:20:11.553 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 25 12:20:11 smithi150 bash[69633]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-h 2024-04-25T12:20:11.664 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T12:20:11.696 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:20:11.696 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T12:20:11.697 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T12:20:11.697 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.k 2024-04-25T12:20:11.916 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:20:11 smithi150 systemd[1]: Stopping Ceph mds.k for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:20:12.417 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:20:12 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-k[61890]: 2024-04-25T12:20:12.012+0000 7f30432da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:20:12.417 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:20:12 smithi150 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-k[61890]: 2024-04-25T12:20:12.012+0000 7f30432da700 -1 mds.k *** got signal Terminated *** 2024-04-25T12:20:15.167 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:20:14 smithi150 podman[69755]: 2024-04-25 12:20:14.912093482 +0000 UTC m=+3.002085046 container died 0cf449dd81c5243949fea98c44bd5f9e04487d522f9f91715d4396e5e21d07e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T12:20:15.573 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:20:15 smithi150 podman[69755]: 2024-04-25 12:20:15.263337752 +0000 UTC m=+3.353329315 container cleanup 0cf449dd81c5243949fea98c44bd5f9e04487d522f9f91715d4396e5e21d07e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T12:20:15.573 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 25 12:20:15 smithi150 bash[69755]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-k 2024-04-25T12:20:15.730 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T12:20:15.760 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:20:15.760 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T12:20:15.761 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T12:20:15.761 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.c 2024-04-25T12:20:16.062 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:20:15 smithi169 systemd[1]: Stopping Ceph mds.c for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:20:16.331 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:20:16 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-c[64164]: 2024-04-25T12:20:16.060+0000 7f4b234a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:20:16.332 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:20:16 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-c[64164]: 2024-04-25T12:20:16.060+0000 7f4b234a5700 -1 mds.c *** got signal Terminated *** 2024-04-25T12:20:19.162 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:20:18 smithi169 podman[71004]: 2024-04-25 12:20:18.909189584 +0000 UTC m=+2.952811760 container died c14b15ecb108210460c1f947f4647727a0c28787ca47bb4fdf5f3f3e925ceea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T12:20:19.606 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:20:19 smithi169 podman[71004]: 2024-04-25 12:20:19.243840191 +0000 UTC m=+3.287462367 container cleanup c14b15ecb108210460c1f947f4647727a0c28787ca47bb4fdf5f3f3e925ceea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T12:20:19.606 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:20:19 smithi169 bash[71004]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-c 2024-04-25T12:20:19.607 INFO:journalctl@ceph.mds.c.smithi169.stdout:Apr 25 12:20:19 smithi169 podman[71030]: 2024-04-25 12:20:19.394814722 +0000 UTC m=+0.480164186 container remove c14b15ecb108210460c1f947f4647727a0c28787ca47bb4fdf5f3f3e925ceea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:20:19.623 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T12:20:19.658 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:20:19.659 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T12:20:19.659 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T12:20:19.659 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.f 2024-04-25T12:20:19.909 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:20:19 smithi169 systemd[1]: Stopping Ceph mds.f for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:20:20.409 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:20:19 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-f[63859]: 2024-04-25T12:20:19.980+0000 7fdd5eed3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:20:20.409 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:20:19 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-f[63859]: 2024-04-25T12:20:19.980+0000 7fdd5eed3700 -1 mds.f *** got signal Terminated *** 2024-04-25T12:20:23.182 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:20:22 smithi169 podman[71136]: 2024-04-25 12:20:22.921817474 +0000 UTC m=+3.039950169 container died 3ac46dda412ee732e92b9022e67fa2ed858d76754923e13d40ad1e4d109795f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T12:20:23.659 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:20:23 smithi169 podman[71136]: 2024-04-25 12:20:23.255345972 +0000 UTC m=+3.373478677 container cleanup 3ac46dda412ee732e92b9022e67fa2ed858d76754923e13d40ad1e4d109795f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-25T12:20:23.660 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:20:23 smithi169 bash[71136]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-f 2024-04-25T12:20:23.660 INFO:journalctl@ceph.mds.f.smithi169.stdout:Apr 25 12:20:23 smithi169 podman[71150]: 2024-04-25 12:20:23.423182764 +0000 UTC m=+0.498399501 container remove 3ac46dda412ee732e92b9022e67fa2ed858d76754923e13d40ad1e4d109795f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T12:20:23.672 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T12:20:23.704 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:20:23.704 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T12:20:23.705 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T12:20:23.705 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.i 2024-04-25T12:20:24.027 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:20:23 smithi169 systemd[1]: Stopping Ceph mds.i for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:20:24.409 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:20:24 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i[64470]: 2024-04-25T12:20:24.025+0000 7fb0039aa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:20:24.409 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:20:24 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i[64470]: 2024-04-25T12:20:24.025+0000 7fb0039aa700 -1 mds.i *** got signal Terminated *** 2024-04-25T12:20:27.159 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:20:26 smithi169 podman[71256]: 2024-04-25 12:20:26.90520184 +0000 UTC m=+2.977251459 container died bdf8b83936cf5911f193fd9d3232ec80244ab110c365ead3236fa5e318386405 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T12:20:27.661 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:20:27 smithi169 podman[71256]: 2024-04-25 12:20:27.25111528 +0000 UTC m=+3.323164896 container cleanup bdf8b83936cf5911f193fd9d3232ec80244ab110c365ead3236fa5e318386405 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T12:20:27.661 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:20:27 smithi169 bash[71256]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i 2024-04-25T12:20:27.661 INFO:journalctl@ceph.mds.i.smithi169.stdout:Apr 25 12:20:27 smithi169 podman[71270]: 2024-04-25 12:20:27.426841194 +0000 UTC m=+0.515816454 container remove bdf8b83936cf5911f193fd9d3232ec80244ab110c365ead3236fa5e318386405 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T12:20:27.683 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T12:20:27.713 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:20:27.713 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T12:20:27.713 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T12:20:27.713 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.l 2024-04-25T12:20:28.122 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:20:27 smithi169 systemd[1]: Stopping Ceph mds.l for be62e290-02fa-11ef-bc93-c7b262605968... 2024-04-25T12:20:28.409 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:20:28 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-l[63541]: 2024-04-25T12:20:28.120+0000 7f6fb1eb5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T12:20:28.409 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:20:28 smithi169 ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-l[63541]: 2024-04-25T12:20:28.120+0000 7f6fb1eb5700 -1 mds.l *** got signal Terminated *** 2024-04-25T12:20:31.306 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:20:30 smithi169 podman[71379]: 2024-04-25 12:20:30.922170238 +0000 UTC m=+3.002080219 container died 49e057e4f52087136c7c249214e8e0ac1caa03b1cbff85bfde28f19f03273be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T12:20:31.659 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:20:31 smithi169 podman[71379]: 2024-04-25 12:20:31.378787359 +0000 UTC m=+3.458697338 container cleanup 49e057e4f52087136c7c249214e8e0ac1caa03b1cbff85bfde28f19f03273be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T12:20:31.659 INFO:journalctl@ceph.mds.l.smithi169.stdout:Apr 25 12:20:31 smithi169 bash[71379]: ceph-be62e290-02fa-11ef-bc93-c7b262605968-mds-l 2024-04-25T12:20:31.795 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-be62e290-02fa-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T12:20:31.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T12:20:31.825 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T12:20:31.825 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid be62e290-02fa-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T12:20:32.026 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T12:20:34.536 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid be62e290-02fa-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T12:20:34.725 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T12:20:37.164 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid be62e290-02fa-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T12:20:37.357 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T12:20:39.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T12:20:39.684 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T12:20:39.716 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T12:20:39.741 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T12:20:39.743 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340/remote/smithi019/crash 2024-04-25T12:20:39.744 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/crash -- . 2024-04-25T12:20:39.795 INFO:teuthology.orchestra.run.smithi019.stderr:tar: /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T12:20:39.795 INFO:teuthology.orchestra.run.smithi019.stderr:tar: Error is not recoverable: exiting now 2024-04-25T12:20:39.797 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340/remote/smithi150/crash 2024-04-25T12:20:39.798 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/crash -- . 2024-04-25T12:20:39.837 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T12:20:39.837 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2024-04-25T12:20:39.839 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340/remote/smithi169/crash 2024-04-25T12:20:39.840 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/crash -- . 2024-04-25T12:20:39.871 INFO:teuthology.orchestra.run.smithi169.stderr:tar: /var/lib/ceph/be62e290-02fa-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T12:20:39.871 INFO:teuthology.orchestra.run.smithi169.stderr:tar: Error is not recoverable: exiting now 2024-04-25T12:20:39.872 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T12:20:39.872 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T12:20:39.909 INFO:tasks.cephadm:Compressing logs... 2024-04-25T12:20:39.910 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T12:20:39.953 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T12:20:39.955 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T12:20:39.977 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.log 2024-04-25T12:20:39.978 INFO:teuthology.orchestra.run.smithi019.stderr:: No such file or directory 2024-04-25T12:20:39.978 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T12:20:39.984 INFO:teuthology.orchestra.run.smithi169.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.log 2024-04-25T12:20:39.984 INFO:teuthology.orchestra.run.smithi169.stderr:: No such file or directory 2024-04-25T12:20:39.984 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T12:20:39.984 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T12:20:39.984 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T12:20:39.984 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T12:20:39.984 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T12:20:39.984 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T12:20:39.985 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T12:20:39.985 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T12:20:39.985 INFO:teuthology.orchestra.run.smithi169.stderr: 87.7% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T12:20:39.985 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mgr.z.log: 90.5% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T12:20:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T12:20:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T12:20:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T12:20:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T12:20:39.986 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T12:20:39.986 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T12:20:39.986 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T12:20:39.986 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T12:20:39.986 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mon.a.log: --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T12:20:39.986 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T12:20:39.986 INFO:teuthology.orchestra.run.smithi019.stderr: 94.6% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T12:20:39.991 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T12:20:39.991 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T12:20:39.991 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T12:20:39.991 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T12:20:39.992 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.11.log: 91.9% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T12:20:39.994 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.log: No such file or directory 2024-04-25T12:20:39.994 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-25T12:20:39.994 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T12:20:39.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T12:20:39.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T12:20:39.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T12:20:39.995 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T12:20:39.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T12:20:39.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T12:20:39.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mgr.y.log: 87.8% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T12:20:39.995 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T12:20:39.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose 90.5% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T12:20:39.996 INFO:teuthology.orchestra.run.smithi150.stderr: -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T12:20:39.996 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T12:20:39.996 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.cephadm.log: /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.10.log: 84.0% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T12:20:39.996 INFO:teuthology.orchestra.run.smithi150.stderr: 91.9% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T12:20:39.997 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T12:20:39.997 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T12:20:39.997 INFO:teuthology.orchestra.run.smithi019.stderr: 84.3% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T12:20:40.000 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T12:20:40.002 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T12:20:40.002 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T12:20:40.008 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T12:20:40.012 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T12:20:40.016 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T12:20:40.020 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T12:20:40.024 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T12:20:40.024 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T12:20:40.027 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T12:20:40.030 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T12:20:40.030 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T12:20:40.031 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T12:20:40.040 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.1% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T12:20:40.040 INFO:teuthology.orchestra.run.smithi169.stderr: 92.9% 93.0% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T12:20:40.040 INFO:teuthology.orchestra.run.smithi169.stderr: -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T12:20:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T12:20:40.042 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T12:20:40.043 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.k.log: 93.0% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T12:20:40.045 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cephadm.log: 94.1% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T12:20:40.045 INFO:teuthology.orchestra.run.smithi150.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T12:20:40.049 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T12:20:40.055 INFO:teuthology.orchestra.run.smithi019.stderr: 93.2% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T12:20:40.076 INFO:teuthology.orchestra.run.smithi169.stderr: 94.4% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T12:20:40.078 INFO:teuthology.orchestra.run.smithi150.stderr: 94.6% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T12:20:40.091 INFO:teuthology.orchestra.run.smithi150.stderr: 94.4% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T12:20:40.177 INFO:teuthology.orchestra.run.smithi169.stderr: 93.4% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T12:20:40.301 INFO:teuthology.orchestra.run.smithi169.stderr: 92.0% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T12:20:40.340 INFO:teuthology.orchestra.run.smithi019.stderr: 90.2% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T12:20:40.381 INFO:teuthology.orchestra.run.smithi150.stderr: 92.1% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T12:20:40.554 INFO:teuthology.orchestra.run.smithi019.stderr: 92.6% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T12:20:40.626 INFO:teuthology.orchestra.run.smithi169.stderr: 92.5% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T12:20:40.682 INFO:teuthology.orchestra.run.smithi150.stderr: 92.3% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T12:20:40.717 INFO:teuthology.orchestra.run.smithi019.stderr: 92.1% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T12:20:40.765 INFO:teuthology.orchestra.run.smithi169.stderr: 92.1% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T12:20:40.779 INFO:teuthology.orchestra.run.smithi019.stderr: 90.4% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T12:20:41.065 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T12:20:41.079 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T12:20:41.102 INFO:teuthology.orchestra.run.smithi169.stderr: 93.2% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T12:20:41.108 INFO:teuthology.orchestra.run.smithi169.stderr: 93.1% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T12:20:41.110 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-04-25T12:20:41.111 INFO:teuthology.orchestra.run.smithi169.stderr:real 0m1.141s 2024-04-25T12:20:41.111 INFO:teuthology.orchestra.run.smithi169.stderr:user 0m4.199s 2024-04-25T12:20:41.111 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m0.164s 2024-04-25T12:20:41.294 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T12:20:41.296 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-25T12:20:41.296 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m1.327s 2024-04-25T12:20:41.296 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m4.658s 2024-04-25T12:20:41.297 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m0.200s 2024-04-25T12:20:41.301 INFO:teuthology.orchestra.run.smithi019.stderr: 93.1% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T12:20:41.391 INFO:teuthology.orchestra.run.smithi019.stderr: 92.8% -- replaced with /var/log/ceph/be62e290-02fa-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T12:20:41.431 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-04-25T12:20:41.431 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m1.427s 2024-04-25T12:20:41.431 INFO:teuthology.orchestra.run.smithi019.stderr:user 0m5.151s 2024-04-25T12:20:41.431 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m0.185s 2024-04-25T12:20:41.431 INFO:tasks.cephadm:Archiving logs... 2024-04-25T12:20:41.432 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340/remote/smithi019/log 2024-04-25T12:20:41.434 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T12:20:42.165 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340/remote/smithi150/log 2024-04-25T12:20:42.166 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T12:20:42.842 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340/remote/smithi169/log 2024-04-25T12:20:42.843 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T12:20:43.446 INFO:tasks.cephadm:Removing cluster... 2024-04-25T12:20:43.446 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid be62e290-02fa-11ef-bc93-c7b262605968 --force 2024-04-25T12:20:43.648 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T12:20:43.919 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid be62e290-02fa-11ef-bc93-c7b262605968 --force 2024-04-25T12:20:44.110 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T12:20:44.421 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid be62e290-02fa-11ef-bc93-c7b262605968 --force 2024-04-25T12:20:44.612 INFO:teuthology.orchestra.run.smithi169.stdout:Deleting cluster with fsid: be62e290-02fa-11ef-bc93-c7b262605968 2024-04-25T12:20:44.862 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T12:20:44.863 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T12:20:44.881 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T12:20:44.896 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T12:20:44.911 INFO:tasks.cephadm:Teardown complete 2024-04-25T12:20:44.912 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T12:20:44.923 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T12:20:44.924 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T12:20:44.926 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T12:20:44.939 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T12:20:45.045 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T12:20:45.055 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T12:20:45.055 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:20:45.058 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:20:45.060 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T12:20:45.072 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:20:45.072 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:20:45.074 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-04-25T12:20:45.075 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:20:45.076 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-04-25T12:20:45.076 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv01.front.sepia.ceph.com 3 6 377 50 +694us[ +686us] +/- 44ms 2024-04-25T12:20:45.076 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 52 -1001us[-1009us] +/- 68ms 2024-04-25T12:20:45.076 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 -100us[ -108us] +/- 76ms 2024-04-25T12:20:45.076 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:20:45.078 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:20:45.078 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-04-25T12:20:45.078 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv01.front.sepia.ceph.com 3 6 377 50 +701us[ +702us] +/- 44ms 2024-04-25T12:20:45.078 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -1004us[-1003us] +/- 68ms 2024-04-25T12:20:45.078 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 -115us[ -114us] +/- 76ms 2024-04-25T12:20:45.078 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:20:45.080 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T12:20:45.080 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-04-25T12:20:45.080 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +670us[ +681us] +/- 44ms 2024-04-25T12:20:45.080 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -1012us[-1012us] +/- 68ms 2024-04-25T12:20:45.080 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 -115us[ -104us] +/- 76ms 2024-04-25T12:20:45.080 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T12:20:45.081 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T12:20:45.090 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T12:20:45.091 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T12:20:45.111 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T12:20:45.200 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T12:20:45.263 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T12:20:45.328 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:20:45.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:20:45.359 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:20:45.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:20:45.386 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T12:20:45.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:20:45.412 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T12:20:45.423 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T12:20:45.452 INFO:teuthology.task.internal:Duration was 1842.118884 seconds 2024-04-25T12:20:45.453 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T12:20:45.486 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T12:20:45.486 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T12:20:45.489 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T12:20:45.491 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T12:20:45.525 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:20:45.527 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:20:45.530 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T12:20:45.869 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T12:20:45.869 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-04-25T12:20:45.870 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T12:20:45.910 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-04-25T12:20:45.910 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T12:20:45.937 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-04-25T12:20:45.937 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T12:20:45.994 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T12:20:45.995 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T12:20:45.997 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T12:20:46.000 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T12:20:46.025 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T12:20:46.026 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T12:20:46.065 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T12:20:46.068 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T12:20:46.461 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T12:20:46.472 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T12:20:46.472 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T12:20:46.496 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T12:20:46.520 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T12:20:46.544 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T12:20:46.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T12:20:46.557 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T12:20:46.562 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T12:20:46.577 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-04-25T12:20:46.582 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-04-25T12:20:46.606 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-04-25T12:20:46.637 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T12:20:46.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:20:46.651 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T12:20:46.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:20:46.665 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T12:20:46.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T12:20:46.691 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T12:20:46.701 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T12:20:46.702 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340/remote/smithi019 2024-04-25T12:20:46.702 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T12:20:46.741 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340/remote/smithi150 2024-04-25T12:20:46.741 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T12:20:46.776 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340/remote/smithi169 2024-04-25T12:20:46.776 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T12:20:46.810 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T12:20:46.810 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T12:20:46.812 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T12:20:46.815 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T12:20:46.858 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T12:20:46.871 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T12:20:46.871 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T12:20:46.888 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T12:20:46.888 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T12:20:46.890 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T12:20:46.892 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T12:20:46.902 INFO:teuthology.orchestra.run.smithi019.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 12:20 /home/ubuntu/cephtest 2024-04-25T12:20:46.904 INFO:teuthology.orchestra.run.smithi150.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 12:20 /home/ubuntu/cephtest 2024-04-25T12:20:46.912 INFO:teuthology.orchestra.run.smithi169.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 12:20 /home/ubuntu/cephtest 2024-04-25T12:20:46.914 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T12:20:46.924 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T12:20:46.988 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T12:20:47.013 DEBUG:teuthology.nuke:shortname: smithi019 2024-04-25T12:20:47.013 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T12:20:47.035 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:44:18.494761', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLIwP2JCCUCyS+uAqsSjt6vJswyEJitzVChNFDE5fYTzW51GPUekb5nTMdmGVdAzHn0BKOp7a5mf2EAtctxln9E='} 2024-04-25T12:20:47.040 DEBUG:teuthology.nuke:shortname: smithi150 2024-04-25T12:20:47.040 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T12:20:47.062 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:44:18.497801', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKKZLkrHzCWxp/EkCzdYWBohZ6e2yWtEhdEyM2rlc1W/W/BoLZ/cnw5kFdvj58/bez8xOVi3OsX8jUAydC+ENkE='} 2024-04-25T12:20:47.065 DEBUG:teuthology.nuke:shortname: smithi169 2024-04-25T12:20:47.066 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T12:20:47.083 INFO:teuthology.orchestra.console.smithi019:Power off 2024-04-25T12:20:47.083 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T12:20:47.094 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673340', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:44:18.496415', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLYgstvRDX4cVmMg8yvJKTYMMudY1X4BY3Bx1vUXf/U5gvj/TeREmehA49fHgA1Nh1OISpZqjMSe65z+dBypyYA='} 2024-04-25T12:20:47.107 DEBUG:teuthology.orchestra.console.smithi019:power off output: Chassis Power Control: Down/Off 2024-04-25T12:20:47.107 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T12:20:47.130 INFO:teuthology.orchestra.console.smithi150:Power off 2024-04-25T12:20:47.130 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T12:20:47.139 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-04-25T12:20:47.151 DEBUG:teuthology.orchestra.console.smithi150:power off output: Chassis Power Control: Down/Off 2024-04-25T12:20:47.151 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T12:20:47.166 INFO:teuthology.orchestra.console.smithi169:Power off 2024-04-25T12:20:47.166 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T12:20:47.174 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-04-25T12:20:47.190 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off 2024-04-25T12:20:47.190 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T12:20:47.212 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-04-25T12:20:51.139 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T12:20:51.176 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T12:20:51.213 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T12:20:51.250 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-04-25T12:20:51.286 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-04-25T12:20:51.322 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-04-25T12:20:55.255 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T12:20:55.287 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T12:20:55.323 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T12:20:55.366 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-04-25T12:20:55.396 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2024-04-25T12:20:55.433 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-04-25T12:20:55.497 INFO:teuthology.orchestra.console.smithi150:Power off completed 2024-04-25T12:20:59.371 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T12:20:59.433 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T12:20:59.482 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2024-04-25T12:20:59.543 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2024-04-25T12:20:59.583 INFO:teuthology.orchestra.console.smithi019:Power off completed 2024-04-25T12:20:59.645 INFO:teuthology.orchestra.console.smithi169:Power off completed 2024-04-25T12:20:59.750 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/norstats}} duration: 1842.1188836097717 failure_reason: '''cluster''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=6087444a5d214a5aa3392b47e5b0fa5c status: fail success: false 2024-04-25T12:20:59.751 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T12:20:59.856 INFO:teuthology.run:FAIL